-- Logs begin at Tue 2022-09-06 15:32:18 UTC, end at Tue 2022-09-06 16:41:45 UTC. -- Sep 06 15:58:59.878909 dv-cinder-852208-18-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 06 15:58:59.888472 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 06 15:58:59.890474 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 6 15:58:59 2022] *** Sep 06 15:58:59.890474 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 06 15:58:59.890474 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 06 15:58:59.890474 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: nodename: dv-cinder-852208-18-iscsi Sep 06 15:58:59.890474 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: machine: x86_64 Sep 06 15:58:59.890474 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: clock source: unix Sep 06 15:58:59.890474 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: pcre jit disabled Sep 06 15:58:59.890474 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: detected number of CPU cores: 4 Sep 06 15:58:59.890474 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: current working directory: / Sep 06 15:58:59.890474 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: detected binary path: /usr/bin/uwsgi-core Sep 06 15:58:59.890474 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: your processes number limit is 47680 Sep 06 15:58:59.890474 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: your memory page size is 4096 bytes Sep 06 15:58:59.891260 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: detected max file descriptor number: 2048 Sep 06 15:58:59.891260 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: lock engine: pthread robust mutexes Sep 06 15:58:59.891260 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: thunder lock: enabled Sep 06 15:58:59.891260 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 06 15:58:59.891260 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 06 15:58:59.929985 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: Python main interpreter initialized at 0x5571a500e9c0 Sep 06 15:58:59.929985 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: python threads support enabled Sep 06 15:58:59.929985 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: your server socket listen backlog is limited to 100 connections Sep 06 15:58:59.929985 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: your mercy for graceful operations on workers is 90 seconds Sep 06 15:58:59.931184 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: mapped 403077 bytes (393 KB) for 2 cores Sep 06 15:58:59.931184 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: *** Operational MODE: preforking *** Sep 06 15:58:59.931410 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: *** uWSGI is running in multiple interpreter mode *** Sep 06 15:58:59.931410 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: spawned uWSGI master process (pid: 81672) Sep 06 15:58:59.931597 dv-cinder-852208-18-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 06 15:58:59.932204 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: spawned uWSGI worker 1 (pid: 81685, cores: 1) Sep 06 15:58:59.932535 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: spawned uWSGI worker 2 (pid: 81686, cores: 1) Sep 06 15:58:59.934431 dv-cinder-852208-18-iscsi devstack@keystone.service[81672]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 06 15:59:04.265265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 15:59:04.265265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: from cryptography.utils import int_from_bytes Sep 06 15:59:04.266103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 15:59:04.266103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: from cryptography.utils import int_from_bytes Sep 06 15:59:04.302303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 15:59:04.302303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: from cryptography.utils import int_from_bytes Sep 06 15:59:04.302983 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 15:59:04.302983 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: from cryptography.utils import int_from_bytes Sep 06 15:59:04.318235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 06 15:59:04.318418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 06 15:59:04.318613 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] command line args: [] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 06 15:59:04.318914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 06 15:59:04.319093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ================================================================================ {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 06 15:59:04.319353 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] admin_token = **** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.319607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] config_dir = [] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.319798 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.319982 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] config_source = [] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.320165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] debug = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.320367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.321841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.321841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.321841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.321841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.321841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] list_limit = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.321841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] log_config_append = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.322181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.322181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] log_dir = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.322334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] log_file = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.322589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.322855 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.323103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.323356 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.323535 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.323713 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.323962 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.324228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.324452 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.324690 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.324924 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.325161 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.325410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.325660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.325946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.326455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.326642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] publish_errors = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.326903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.327140 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.327320 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.327570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.327807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.328047 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] standard_threads = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.328282 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.328528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.328706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.328882 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] use_journal = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.329059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] use_json = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.329236 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] use_stderr = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.329412 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] use_syslog = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.329645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.329917 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.330253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.330410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.330673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.330933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.331175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.331475 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.331692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.331922 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.332174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.332427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.332666 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.332917 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.333156 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.333406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.333656 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.333895 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.334152 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.334408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.334673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.334918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.335173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.335435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.335690 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.335940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.336177 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.336457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.336711 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.336991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.337177 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.337429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.337667 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.337896 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.338146 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.338400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.338642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.338923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.339159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.339396 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.339636 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.339870 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.340105 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.340331 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.340566 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.340803 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.341054 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.341296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.341536 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.341771 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.342011 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.342250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.342500 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.342756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.342998 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.343239 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.343476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.343741 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.343952 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.344184 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.344435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.344676 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.344923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.345151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.345416 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.345658 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.345893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.346134 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.346375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.346650 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.346914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.347166 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.347414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.347655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.347897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.348137 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.348387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.348628 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.348869 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.349114 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.349353 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.349602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.349842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.350082 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.350319 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.350555 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.350819 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.351094 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.351346 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.351494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 06 15:59:04.351627 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 06 15:59:04.351752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.351867 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] command line args: [] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 06 15:59:04.352554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.352695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 06 15:59:04.352802 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.352897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ================================================================================ {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 06 15:59:04.353013 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.353112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] admin_token = **** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.353237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.353307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] config_dir = [] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.353513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.353660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.353749 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] config_source = [] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.353843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] debug = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.353924 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.354055 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.354160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.354296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.354373 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.354531 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.354708 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.354785 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.354867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.355018 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.355094 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.355254 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] list_limit = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.355336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.355436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] log_config_append = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.355510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.355622 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.355764 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.355858 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] log_dir = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.355964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] log_file = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.356041 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.356203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.356289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.356454 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.356531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.356704 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.356788 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.356937 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] auth.external = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.357016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.357119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.357201 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.357298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.357457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.357556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.357700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.357792 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.357923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] auth.password = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.358000 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.358158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] auth.token = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.358236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.358401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.358478 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.358642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.358753 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.358916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.358996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.359338 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.359426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.359487 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.359543 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.359644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.359879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.359957 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.360137 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] credential.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.360230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] publish_errors = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.360364 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.360455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.360606 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.360691 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.360783 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.360866 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.361026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.361099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.361268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.361353 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.361509 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] standard_threads = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.361606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.361745 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.361823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.361996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.362099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.362259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.362462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.362548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.362654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] use_journal = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.362827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.362928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] use_json = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.363017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] use_stderr = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.363094 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.363227 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] use_syslog = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.363360 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.363448 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 15:59:04.363632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.363762 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.363837 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.363908 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.364142 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] federation.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.364263 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.364373 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.364472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.364598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.364707 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.364808 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.364908 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.365005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.365102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.365237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.365348 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.365479 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.365579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.365735 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.365805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.365977 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.366059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.366411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.366504 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.366580 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.366716 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.366792 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.366874 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.366947 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.367010 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.367206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.367292 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.367456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.367542 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.368338 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.368515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.368736 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.368839 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.369018 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.369152 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.369265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.369395 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.369528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.369638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.369801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.369887 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.370070 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.370185 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.370326 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.370407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.370589 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.370775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.370878 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.370968 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.371043 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.371112 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.371331 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.371427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.371543 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.371618 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.371788 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.371868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.372022 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.372100 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.372261 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.372340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.372513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.372598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.372765 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.372842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.373004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.373084 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.373246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.373323 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.373480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.373557 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.373719 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.373797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.373963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.374070 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.374200 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.374281 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.374438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.374522 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.374685 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.374767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.374927 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.375004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.375173 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.375250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.375411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.375491 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.375654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.375739 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.375896 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.375982 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.376128 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.376208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.376370 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.376453 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.376601 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.376678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.376843 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.376923 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.377078 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.377155 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.377320 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.377399 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.377560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.377647 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.377799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.377875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.378039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.378117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.378282 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.378431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.378509 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.378691 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.378775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.378944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.379040 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.379200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.379291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.379454 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.379533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.379698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.379789 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.379942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.380022 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.380177 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.380254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.380410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.380495 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.380656 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.380735 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.380919 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.380997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.381156 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.381243 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.381408 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.381494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.381658 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.381735 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.381899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.381984 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.382137 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.382214 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.382371 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.382451 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.382612 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.382813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.382951 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.383060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.383175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.383286 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.384039 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.384193 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.384328 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.384443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.384559 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.384687 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.384802 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.385020 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.385138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.385216 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.385276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.385357 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.385439 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.385519 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.385592 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.385662 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.385732 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.385802 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.385880 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.385953 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.386024 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.386093 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.386163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.386251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.386375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.386479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.386574 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.386680 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.386756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.386888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.386981 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.387130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.387208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.387372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.387450 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.387611 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.387697 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.387847 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.387925 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.388085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.388162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.388324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.388409 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.388594 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.388683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.388806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.388875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.389048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.389125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.389366 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.389537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] resource.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.389633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.389799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.389976 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.390063 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.390299 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.390423 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.390536 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] auth.external = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.390799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.390888 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.391108 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.391347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.391426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.391583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] auth.password = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.391819 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.391898 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] auth.token = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.392063 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.392279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.392357 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.392544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.392772 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.392850 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.393023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.393241 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.393320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.393500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.393697 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.393775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] credential.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.393983 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.394171 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] role.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.394248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.394480 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.394651 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] role.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.394797 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.395010 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.395152 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.395289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.395500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.395643 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.395754 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.395991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.396097 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.396248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.396542 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.396626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.396797 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.397050 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.397131 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.397308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.397530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.397613 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.397798 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] federation.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.398017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.398097 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.398279 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.398508 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.398602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.398784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.399155 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.399617 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.399617 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.399760 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.399832 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.399914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.399990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.400172 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.400415 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.400493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.400668 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.400906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.400987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.401161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.401384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.401463 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.401643 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.401885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.402136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.402214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.402375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.402620 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.402850 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.402930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.403126 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.403206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.403375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.403591 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.403669 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.403927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.404198 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.404413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.404633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.404711 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.404922 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.405013 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.405179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.405284 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.405442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.405520 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.405702 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.405804 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.405993 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.406085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.406236 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.406380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.406539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.406701 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.406834 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.406965 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.407107 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.407237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.407364 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.407503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.407632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.407761 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.407889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.408016 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.408161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.408292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] token.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.408425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.408556 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.408688 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.408820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.408922 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.408998 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.409068 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.409139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.409208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.409277 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.409356 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.409429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.409531 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.409611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.409770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.409848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.410016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.410095 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.410268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.410368 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.410498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.410600 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.410765 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.410847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.411015 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.411092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.411262 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.411340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.411490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.411565 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.411753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.411839 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.412006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.412082 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.412245 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.412322 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.connection = **** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.412483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.412562 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.412738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.412816 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.412983 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.413067 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.413226 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.413304 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.413464 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.413540 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.413704 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.413781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.413940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.414019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.414181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.414287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.414954 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.415104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.415258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.415370 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.415466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.415548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.415663 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.415744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.415917 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.415998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.416158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.416236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.416403 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.416481 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.416637 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.416717 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.416883 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.416961 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.417124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.417210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.417348 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 06 15:59:04.417428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.417632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.417887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.418140 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.418394 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.418649 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.418912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.419167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.419409 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.419655 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.419894 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.420134 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.420384 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.420775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.421020 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.421285 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.421524 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.421782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.422026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.422271 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.422514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.422793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.423042 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.423281 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.423525 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.423774 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] resource.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.424031 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.424284 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.424783 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.425040 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.425298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.425541 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.425796 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.426038 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.426284 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.426527 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] role.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.426808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] role.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.427046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.427303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.427558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.427814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.428068 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.428321 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.428575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.428825 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.429077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.429315 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.429568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.429822 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.430167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.431405 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.431679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.431932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.432192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.432451 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.432707 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.433079 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.433209 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.433476 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.433730 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.433974 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.434813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.435063 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.435315 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.435580 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.435848 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.436108 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.436358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.436605 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.436848 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.437095 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] token.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.437340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.437597 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.437843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.438108 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.438365 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.438610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.438889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.439143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.439402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.439650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.439900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.440145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.440389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.440673 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.440910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.441159 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.441434 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.441703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.connection = **** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.441953 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.442215 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.442458 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.442729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.442972 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.443215 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.443460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.443699 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.443948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.444189 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.444436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.444678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.444932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.445172 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.445421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.445659 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.446010 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.446506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 15:59:04.446506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 06 15:59:04.453441 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.453669 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.453903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.462454 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.471980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.472217 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.472450 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.481447 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.491013 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.491250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.500570 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.500808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.546605 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.549868 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.596885 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.597185 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.598562 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.598835 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.757540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.757926 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.758060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.758179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.758900 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.759020 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.759172 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.759430 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.759564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.759690 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.760010 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.760300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.783990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.784291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.784425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.784556 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.784688 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.784838 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.784961 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.785086 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.785207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.785321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.785451 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.785559 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.785665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.785665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.797450 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.797611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.811568 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.811798 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.811889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.812027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.812111 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.812254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.812339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.812477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.812559 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.812695 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.812785 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.812909 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.812989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.813135 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 15:59:04.830530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.830765 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 15:59:04.830962 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.831113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.831196 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 15:59:04.831315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 15:59:04.831513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.831618 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.831722 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 15:59:04.831840 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.832003 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.832103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.832277 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.832391 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.832538 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.832799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.842007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.842290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.842511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.842610 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.842822 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.842904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.843099 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.843388 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.849815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.850381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.853375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.853663 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.853950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.854132 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.854238 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.854429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.854524 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.854743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.854839 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.855037 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.855118 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.855329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.855408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.855599 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.855687 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.855876 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.855954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.856151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.856248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.856427 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.856712 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.856989 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.879431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.880883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.880986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.881178 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.882374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.882692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.885860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.886149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.886439 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.887297 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.887593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.887884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.892812 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.893106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.894372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.894685 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.898650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.899000 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.899137 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.899294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.899427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.899723 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.899812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.900009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.900288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.900568 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.900844 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.901116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.901880 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.902181 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.902475 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.902797 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.917209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.917503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.918995 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.919291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.921373 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.923163 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.924907 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.925205 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.926551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.926891 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.929435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.931384 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.932816 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.934518 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.936057 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.936361 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.936653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.936949 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.937244 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.937536 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.937670 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.937977 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.938270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.938725 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.939120 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.939423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.953781 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.954075 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.954346 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.954609 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.956001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.956300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.956574 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.956845 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.962007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.963605 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.964388 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.965142 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.965420 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.966074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.967649 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.967944 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.968104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.968395 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.968682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.968966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.969373 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.970933 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.971229 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.971519 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.971810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.972108 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.980973 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.981267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.981554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.981840 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.982227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.982512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.982820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.983471 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:04.983769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.984060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.984350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.984643 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.984924 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.985205 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:04.998538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:05.000848 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:05.001468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:05.003951 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:05.004366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.005793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.006904 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.007310 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:05.007598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.007997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.008945 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.010436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:05.010749 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.011046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.015418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.016873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:05.017580 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.019093 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:05.019787 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.020187 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.020476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.020758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.021980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.022281 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.022571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.022878 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.029158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:05.029453 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.029750 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.031436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:05.031741 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.032023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.036738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:05.037026 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.037310 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.037600 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.037887 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.038163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.038437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.038735 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.038958 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 15:59:05.039053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.039248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.039338 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.039559 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.039839 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.039929 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.040194 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.040282 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.040465 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.040554 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.040730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.040810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.041013 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.041290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.041593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.041871 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.042144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.042414 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.042698 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 15:59:05.075981 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5571a500e9c0 pid: 81686 (default app) Sep 06 15:59:05.078170 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5571a500e9c0 pid: 81685 (default app) Sep 06 15:59:05.079339 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3afa94-a670-4c7e-95f5-b1f45e75f9b1 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:05.079631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3afa94-a670-4c7e-95f5-b1f45e75f9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:05.079912 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3afa94-a670-4c7e-95f5-b1f45e75f9b1 None None] PATH_INFO: `/v3` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:05.084072 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Tue Sep 6 15:59:05 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:11.884773 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a05b6-089a-45cb-87cf-e23e29dfd3be None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:11.885346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a05b6-089a-45cb-87cf-e23e29dfd3be None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:11.885346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a05b6-089a-45cb-87cf-e23e29dfd3be None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:11.889449 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:11.895150 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32791f8-7b39-4a26-8d92-e0bf11e71323 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:11.895704 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32791f8-7b39-4a26-8d92-e0bf11e71323 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:11.895704 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32791f8-7b39-4a26-8d92-e0bf11e71323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:11.967437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: INFO dbcounter [None req-e32791f8-7b39-4a26-8d92-e0bf11e71323 None None] Registered counter for database keystone Sep 06 15:59:11.986684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG oslo_db.sqlalchemy.engines [None req-e32791f8-7b39-4a26-8d92-e0bf11e71323 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81686) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 06 15:59:11.993408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writer thread running {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 06 15:59:12.173597 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG passlib.handlers.bcrypt [None req-e32791f8-7b39-4a26-8d92-e0bf11e71323 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81686) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 06 15:59:12.173907 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG passlib.handlers.bcrypt [None req-e32791f8-7b39-4a26-8d92-e0bf11e71323 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81686) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 06 15:59:12.236437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e32791f8-7b39-4a26-8d92-e0bf11e71323 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:12.238321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e32791f8-7b39-4a26-8d92-e0bf11e71323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:12.377598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 06 15:59:12.688550 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a05b6-089a-45cb-87cf-e23e29dfd3be None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:12.772353 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: INFO dbcounter [None req-cc2a05b6-089a-45cb-87cf-e23e29dfd3be None None] Registered counter for database keystone Sep 06 15:59:12.795103 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG oslo_db.sqlalchemy.engines [None req-cc2a05b6-089a-45cb-87cf-e23e29dfd3be None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81685) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 06 15:59:12.803062 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writer thread running {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 06 15:59:12.931761 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a05b6-089a-45cb-87cf-e23e29dfd3be None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:12.936255 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabda5c-9e9f-4d36-adc9-37308363c8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:12.936909 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabda5c-9e9f-4d36-adc9-37308363c8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:12.937217 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabda5c-9e9f-4d36-adc9-37308363c8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:12.937513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabda5c-9e9f-4d36-adc9-37308363c8d1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:12.948208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aabda5c-9e9f-4d36-adc9-37308363c8d1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:13.001826 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7aabda5c-9e9f-4d36-adc9-37308363c8d1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 06 15:59:13.002160 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aabda5c-9e9f-4d36-adc9-37308363c8d1 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:13.013297 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Tue Sep 6 15:59:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 15:59:13.019926 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32791f8-7b39-4a26-8d92-e0bf11e71323 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:13.035448 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32791f8-7b39-4a26-8d92-e0bf11e71323 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:13.040007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bf54b-0aa3-4d86-8827-e99268f79da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:13.041139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bf54b-0aa3-4d86-8827-e99268f79da2 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:13.041422 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bf54b-0aa3-4d86-8827-e99268f79da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:13.041935 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bf54b-0aa3-4d86-8827-e99268f79da2 admin admin] PATH_INFO: `/v3/projects/dcbe5a1252a54f0a914b3779a09b044d` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:13.043690 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bf54b-0aa3-4d86-8827-e99268f79da2 admin admin] RBAC: Authorizing `identity:get_project(project_id=dcbe5a1252a54f0a914b3779a09b044d)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:13.100246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a25bf54b-0aa3-4d86-8827-e99268f79da2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 06 15:59:13.100583 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bf54b-0aa3-4d86-8827-e99268f79da2 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:13.106053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Tue Sep 6 15:59:13 2022] GET /identity/v3/projects/dcbe5a1252a54f0a914b3779a09b044d => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:15.022349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec38adf-d050-4f3e-8582-e5828d203688 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:15.022893 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec38adf-d050-4f3e-8582-e5828d203688 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:15.022893 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec38adf-d050-4f3e-8582-e5828d203688 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:15.027445 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:15.033042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e454f-3803-463d-9799-4b193d9d9464 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:15.033458 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e454f-3803-463d-9799-4b193d9d9464 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:15.033605 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e454f-3803-463d-9799-4b193d9d9464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:15.055875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f81e454f-3803-463d-9799-4b193d9d9464 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:15.057466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f81e454f-3803-463d-9799-4b193d9d9464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:15.065611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 06 15:59:15.335624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec38adf-d050-4f3e-8582-e5828d203688 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:15.348029 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec38adf-d050-4f3e-8582-e5828d203688 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:15.353059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d6809-aa00-429c-a3a5-785ca2d817a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:15.353663 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d6809-aa00-429c-a3a5-785ca2d817a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:15.353946 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d6809-aa00-429c-a3a5-785ca2d817a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:15.354223 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d6809-aa00-429c-a3a5-785ca2d817a0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:15.362401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0d6809-aa00-429c-a3a5-785ca2d817a0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:15.364219 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0d6809-aa00-429c-a3a5-785ca2d817a0 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:15.374596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Tue Sep 6 15:59:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 15:59:15.380319 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e454f-3803-463d-9799-4b193d9d9464 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:15.389741 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e454f-3803-463d-9799-4b193d9d9464 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:15.394091 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223ad36-b165-48d9-bc1b-9b27b535d805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:15.394758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223ad36-b165-48d9-bc1b-9b27b535d805 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:15.395039 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223ad36-b165-48d9-bc1b-9b27b535d805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:15.395308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223ad36-b165-48d9-bc1b-9b27b535d805 admin admin] PATH_INFO: `/v3/users/feda116f177c415fa220633084d10d2f` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:15.396627 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5223ad36-b165-48d9-bc1b-9b27b535d805 admin admin] RBAC: Authorizing `identity:get_user(user_id=feda116f177c415fa220633084d10d2f)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:15.398477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5223ad36-b165-48d9-bc1b-9b27b535d805 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:15.400329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Tue Sep 6 15:59:15 2022] GET /identity/v3/users/feda116f177c415fa220633084d10d2f => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:17.376864 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31f2d4-9cb3-4163-a224-edf7ce92de64 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:17.377413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31f2d4-9cb3-4163-a224-edf7ce92de64 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:17.377413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31f2d4-9cb3-4163-a224-edf7ce92de64 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:17.378807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:17.384653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6665e-a0b4-43df-8132-f94cc50d8889 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:17.385083 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6665e-a0b4-43df-8132-f94cc50d8889 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:17.385190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6665e-a0b4-43df-8132-f94cc50d8889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:17.403469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-fee6665e-a0b4-43df-8132-f94cc50d8889 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:17.404631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fee6665e-a0b4-43df-8132-f94cc50d8889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:17.447456 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ec2c6-d8e2-48ad-970b-8a0541184827 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:17.447734 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ec2c6-d8e2-48ad-970b-8a0541184827 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:17.449462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ec2c6-d8e2-48ad-970b-8a0541184827 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:17.450844 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 15:59:17.456278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5d181-d4f8-4d96-a2d1-6e381cf598b6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:17.456566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5d181-d4f8-4d96-a2d1-6e381cf598b6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:17.456845 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5d181-d4f8-4d96-a2d1-6e381cf598b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:17.502321 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG passlib.handlers.bcrypt [None req-2cc5d181-d4f8-4d96-a2d1-6e381cf598b6 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81685) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 06 15:59:17.502737 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG passlib.handlers.bcrypt [None req-2cc5d181-d4f8-4d96-a2d1-6e381cf598b6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81685) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 06 15:59:17.522336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 15:59:17.553416 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2cc5d181-d4f8-4d96-a2d1-6e381cf598b6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:17.556118 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2cc5d181-d4f8-4d96-a2d1-6e381cf598b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:17.671566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 15:59:17.800569 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6665e-a0b4-43df-8132-f94cc50d8889 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:17.832220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:17.918367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6665e-a0b4-43df-8132-f94cc50d8889 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:17.945898 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5d181-d4f8-4d96-a2d1-6e381cf598b6 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:17.956957 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189114be-43e1-44b6-bf40-a88917f1bdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:17.957631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189114be-43e1-44b6-bf40-a88917f1bdee None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:17.957882 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189114be-43e1-44b6-bf40-a88917f1bdee None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:17.958159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189114be-43e1-44b6-bf40-a88917f1bdee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:17.971367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189114be-43e1-44b6-bf40-a88917f1bdee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:17.982933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-189114be-43e1-44b6-bf40-a88917f1bdee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 15:59:17.982933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:17.982933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:17.982933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:17.982933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:17.982933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:17.982933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:17.982933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:17.982933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:17.982933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:17.982933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:17.982933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:17.982933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:17.982933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:17.982933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:17.982933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:17.984093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:17.984093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:17.984093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:17.984093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:17.984093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:17.984093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:17.984093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:17.984093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:17.984093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:17.984093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:17.984093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:17.984093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:17.984093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:17.984093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:17.984093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:17.984093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:17.985179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:17.985179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:17.985179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:17.985179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:17.985179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:17.985179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:17.985179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:17.985179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:17.985179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:17.985179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:17.985179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 15:59:17.985179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 15:59:17.985179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 15:59:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:17.998046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189114be-43e1-44b6-bf40-a88917f1bdee None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:18.083098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5d181-d4f8-4d96-a2d1-6e381cf598b6 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:18.085721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:18.093342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:18.118807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189114be-43e1-44b6-bf40-a88917f1bdee None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:18.126788 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a019ca-053a-4cc6-b5a4-b9da3ca86753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:18.127672 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a019ca-053a-4cc6-b5a4-b9da3ca86753 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:18.127953 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a019ca-053a-4cc6-b5a4-b9da3ca86753 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:18.128224 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a019ca-053a-4cc6-b5a4-b9da3ca86753 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:18.155664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8de261-8b18-44cc-9749-8e51949244d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:18.156272 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8de261-8b18-44cc-9749-8e51949244d1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:18.156559 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8de261-8b18-44cc-9749-8e51949244d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:18.156827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8de261-8b18-44cc-9749-8e51949244d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:18.157586 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8de261-8b18-44cc-9749-8e51949244d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:18.157911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8de261-8b18-44cc-9749-8e51949244d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:18.159368 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8de261-8b18-44cc-9749-8e51949244d1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:18.173614 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 15:59:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 15:59:18.179493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8de261-8b18-44cc-9749-8e51949244d1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:18.208284 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a019ca-053a-4cc6-b5a4-b9da3ca86753 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:18.209729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a019ca-053a-4cc6-b5a4-b9da3ca86753 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:18.315310 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8de261-8b18-44cc-9749-8e51949244d1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:18.319738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 8/16] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 15:59:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 15:59:18.326265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a019ca-053a-4cc6-b5a4-b9da3ca86753 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:18.329583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:18.362897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7b798-bab1-4e62-b3a3-761f6a122a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:18.363571 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7b798-bab1-4e62-b3a3-761f6a122a29 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:18.363857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7b798-bab1-4e62-b3a3-761f6a122a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:18.364288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7b798-bab1-4e62-b3a3-761f6a122a29 None admin] PATH_INFO: `/v3/users/feda116f177c415fa220633084d10d2f` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:18.365743 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7b798-bab1-4e62-b3a3-761f6a122a29 None admin] RBAC: Authorizing `identity:get_user(user_id=feda116f177c415fa220633084d10d2f)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:18.367326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7b798-bab1-4e62-b3a3-761f6a122a29 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:18.369729 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:18 2022] GET /identity/v3/users/feda116f177c415fa220633084d10d2f => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 15:59:18.375627 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7b798-bab1-4e62-b3a3-761f6a122a29 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:18.396451 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:18.450431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a019ca-053a-4cc6-b5a4-b9da3ca86753 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:18.492585 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ca181-a1ed-496f-9a4b-791cbedd6041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:18.493031 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ca181-a1ed-496f-9a4b-791cbedd6041 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:18.493612 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ca181-a1ed-496f-9a4b-791cbedd6041 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:18.493612 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ca181-a1ed-496f-9a4b-791cbedd6041 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:18.496292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7b798-bab1-4e62-b3a3-761f6a122a29 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:18.514944 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ca181-a1ed-496f-9a4b-791cbedd6041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:18.516454 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ca181-a1ed-496f-9a4b-791cbedd6041 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:18.538075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-c78ca181-a1ed-496f-9a4b-791cbedd6041 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 15:59:18.538075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:18.538075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:18.538075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:18.538075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:18.538075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:18.538075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:18.538075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:18.538075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:18.538075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:18.538075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:18.538075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:18.538075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 15:59:18.538075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 15:59:18.538075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 15:59:18.538075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 15:59:18.539090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:18.539090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:18.539090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:18.539090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:18.539090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:18.539090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:18.539090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:18.539090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:18.539090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:18.539090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:18.539090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:18.539090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:18.539090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:18.539090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:18.539090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:18.539090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:18.539090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 15:59:18.539886 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 15:59:18.539886 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 15:59:18.539886 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 15:59:18.540103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fce20-129c-43ae-aff5-fe8dad8e3731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:18.540245 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 9/18] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 15:59:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 15:59:18.540838 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fce20-129c-43ae-aff5-fe8dad8e3731 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:18.541297 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fce20-129c-43ae-aff5-fe8dad8e3731 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:18.541661 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fce20-129c-43ae-aff5-fe8dad8e3731 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:18.543277 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fce20-129c-43ae-aff5-fe8dad8e3731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:18.545414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fce20-129c-43ae-aff5-fe8dad8e3731 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:18.545590 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ca181-a1ed-496f-9a4b-791cbedd6041 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:18.547769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 10/19] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 15:59:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 15:59:18.553989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fce20-129c-43ae-aff5-fe8dad8e3731 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:18.620364 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:18.644263 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:18.665671 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fce20-129c-43ae-aff5-fe8dad8e3731 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:18.685982 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ca181-a1ed-496f-9a4b-791cbedd6041 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:18.707227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5cce6-f2e5-4fb4-9294-fb45c4c79939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:18.708216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5cce6-f2e5-4fb4-9294-fb45c4c79939 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:18.708630 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5cce6-f2e5-4fb4-9294-fb45c4c79939 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:18.708911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5cce6-f2e5-4fb4-9294-fb45c4c79939 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:18.710059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5cce6-f2e5-4fb4-9294-fb45c4c79939 None admin] RBAC: Adding query filter params (role.id=ba05677650e046a48d5f19440d48fbf2, scope.domain.id=default, user.id=feda116f177c415fa220633084d10d2f) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:18.710579 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5cce6-f2e5-4fb4-9294-fb45c4c79939 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:18.713056 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5cce6-f2e5-4fb4-9294-fb45c4c79939 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:18.720780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 11/20] 10.222.0.26 () {56 vars in 1322 bytes} [Tue Sep 6 15:59:18 2022] GET /identity/v3/role_assignments?user.id=feda116f177c415fa220633084d10d2f&scope.domain.id=default&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 233 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 15:59:18.724954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e1eb34-6e25-4166-a250-4f2594bb31f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:18.725691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1eb34-6e25-4166-a250-4f2594bb31f4 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:18.726029 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1eb34-6e25-4166-a250-4f2594bb31f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:18.726356 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1eb34-6e25-4166-a250-4f2594bb31f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:18.727310 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e1eb34-6e25-4166-a250-4f2594bb31f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:18.727762 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e1eb34-6e25-4166-a250-4f2594bb31f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:18.729304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e1eb34-6e25-4166-a250-4f2594bb31f4 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:18.753190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 10/21] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 15:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:21.483683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b3cc6-1825-4b20-97cc-a52c2a1e8b5e None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:21.484711 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b3cc6-1825-4b20-97cc-a52c2a1e8b5e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:21.485201 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b3cc6-1825-4b20-97cc-a52c2a1e8b5e None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:21.487007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 12/22] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:21.505077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee5e53-9190-4323-a72b-6b51465b6fa1 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:21.505574 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee5e53-9190-4323-a72b-6b51465b6fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:21.505646 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee5e53-9190-4323-a72b-6b51465b6fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:21.536410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-adee5e53-9190-4323-a72b-6b51465b6fa1 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:21.537635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-adee5e53-9190-4323-a72b-6b51465b6fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:21.641095 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 11/23] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 15:59:21.809366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63908d-a744-4015-859b-a462ee993c68 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:21.810016 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63908d-a744-4015-859b-a462ee993c68 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:21.810684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63908d-a744-4015-859b-a462ee993c68 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:21.812339 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 13/24] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 15:59:21.819324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da6efe-b89f-4e0f-b32e-2fdf466a7ec3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:21.819825 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da6efe-b89f-4e0f-b32e-2fdf466a7ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:21.820891 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da6efe-b89f-4e0f-b32e-2fdf466a7ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:21.851975 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b5da6efe-b89f-4e0f-b32e-2fdf466a7ec3 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:21.853560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b5da6efe-b89f-4e0f-b32e-2fdf466a7ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:21.921108 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:21.955281 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 12/25] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:22.268869 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1ac72-36f6-478e-b046-05ad78d8309e None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:22.269332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1ac72-36f6-478e-b046-05ad78d8309e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:22.269486 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1ac72-36f6-478e-b046-05ad78d8309e None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:22.271109 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 14/26] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:22.279033 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa463717-9beb-49dc-bfdf-69f0486924d6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:22.279376 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa463717-9beb-49dc-bfdf-69f0486924d6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:22.279605 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa463717-9beb-49dc-bfdf-69f0486924d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:22.307770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-aa463717-9beb-49dc-bfdf-69f0486924d6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:22.309011 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-aa463717-9beb-49dc-bfdf-69f0486924d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:22.376281 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1ac72-36f6-478e-b046-05ad78d8309e None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:22.439279 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 13/27] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:22.442012 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ff09b-a6b7-4258-98c1-c0044114a9c3 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:22.442301 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ff09b-a6b7-4258-98c1-c0044114a9c3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:22.442577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ff09b-a6b7-4258-98c1-c0044114a9c3 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:22.446556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 14/28] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:22.451055 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db9ad5-6006-4feb-91a0-871147148d01 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:22.451360 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db9ad5-6006-4feb-91a0-871147148d01 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:22.451623 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db9ad5-6006-4feb-91a0-871147148d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:22.488813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-00db9ad5-6006-4feb-91a0-871147148d01 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:22.490850 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-00db9ad5-6006-4feb-91a0-871147148d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:22.541393 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1ac72-36f6-478e-b046-05ad78d8309e None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:22.631912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 15/29] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:22.633844 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db9ad5-6006-4feb-91a0-871147148d01 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:22.634710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9360f-c32d-4f14-8b42-7e6a23d887cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:22.635327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9360f-c32d-4f14-8b42-7e6a23d887cb None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:22.635596 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9360f-c32d-4f14-8b42-7e6a23d887cb None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:22.635868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9360f-c32d-4f14-8b42-7e6a23d887cb None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:22.636682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9360f-c32d-4f14-8b42-7e6a23d887cb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:22.638154 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9360f-c32d-4f14-8b42-7e6a23d887cb None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:22.762210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:22.823654 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 15/30] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Sep 6 15:59:22 2022] POST /identity/v3/roles => generated 219 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 15:59:22.908624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db9ad5-6006-4feb-91a0-871147148d01 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:22.956588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27d3dd-2738-4c45-a20a-ac182809e90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:22.957278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27d3dd-2738-4c45-a20a-ac182809e90e None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:22.957570 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27d3dd-2738-4c45-a20a-ac182809e90e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:22.957848 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27d3dd-2738-4c45-a20a-ac182809e90e None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:22.958542 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27d3dd-2738-4c45-a20a-ac182809e90e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:22.959997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27d3dd-2738-4c45-a20a-ac182809e90e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:23.061908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9360f-c32d-4f14-8b42-7e6a23d887cb None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:23.066474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:23.161066 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 16/31] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Sep 6 15:59:22 2022] POST /identity/v3/roles => generated 225 bytes in 529 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 15:59:23.162201 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27d3dd-2738-4c45-a20a-ac182809e90e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:23.333873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9360f-c32d-4f14-8b42-7e6a23d887cb None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:23.405192 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8d688-6451-472c-a317-0105e4ed4344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:23.406238 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8d688-6451-472c-a317-0105e4ed4344 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:23.406729 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8d688-6451-472c-a317-0105e4ed4344 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:23.407201 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8d688-6451-472c-a317-0105e4ed4344 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:23.427991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8d688-6451-472c-a317-0105e4ed4344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:23.440631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-05f8d688-6451-472c-a317-0105e4ed4344 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 15:59:23.440631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:23.440631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:23.440631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:23.440631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:23.440631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:23.440631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:23.440631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:23.440631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:23.440631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:23.440631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:23.440631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:23.440631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:23.440631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:23.440631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:23.440631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:23.441538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:23.441538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:23.441538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:23.441538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:23.441538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:23.441538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:23.441538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:23.441538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:23.441538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:23.441538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:23.441538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:23.441538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:23.441538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:23.441538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:23.441538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:23.441538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:23.442276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:23.442276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:23.442276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:23.442276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:23.442276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:23.442276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:23.442276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:23.442276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:23.442276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:23.442276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:23.442276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 15:59:23.442276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 15:59:23.443508 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 16/32] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 15:59:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:23.443572 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27d3dd-2738-4c45-a20a-ac182809e90e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:23.452523 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8d688-6451-472c-a317-0105e4ed4344 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:23.471837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:23.518563 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440e212-fdee-4ea2-ae11-fb2aeba00d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:23.521528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440e212-fdee-4ea2-ae11-fb2aeba00d26 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:23.522270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440e212-fdee-4ea2-ae11-fb2aeba00d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:23.523142 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440e212-fdee-4ea2-ae11-fb2aeba00d26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:23.547967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440e212-fdee-4ea2-ae11-fb2aeba00d26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:23.552465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440e212-fdee-4ea2-ae11-fb2aeba00d26 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:23.579425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-e440e212-fdee-4ea2-ae11-fb2aeba00d26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 15:59:23.579425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:23.579425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:23.579425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:23.579425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:23.579425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:23.579425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:23.579425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:23.579425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:23.579425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:23.579425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:23.579425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:23.579425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 15:59:23.579425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 15:59:23.579425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 15:59:23.579425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 15:59:23.581502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:23.581502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:23.581502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:23.581502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:23.581502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:23.581502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:23.581502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:23.581502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:23.581502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:23.581502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:23.581502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:23.581502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:23.581502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:23.581502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:23.581502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:23.581502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:23.581502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 15:59:23.582998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 15:59:23.582998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 15:59:23.582998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 15:59:23.582998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 17/33] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 15:59:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:23.582998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4100418-c15c-4332-8c14-56a1eb8586e1 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:23.582998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4100418-c15c-4332-8c14-56a1eb8586e1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:23.584392 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:23.587122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4100418-c15c-4332-8c14-56a1eb8586e1 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:23.591274 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 18/34] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:23 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:23.592560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4100418-c15c-4332-8c14-56a1eb8586e1 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:23.642711 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8d688-6451-472c-a317-0105e4ed4344 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:23.691377 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc2a0b-10d5-4a65-9dfd-11a7d1ffd98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:23.692269 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc2a0b-10d5-4a65-9dfd-11a7d1ffd98f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:23.692786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc2a0b-10d5-4a65-9dfd-11a7d1ffd98f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:23.693169 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc2a0b-10d5-4a65-9dfd-11a7d1ffd98f None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:23.694204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc2a0b-10d5-4a65-9dfd-11a7d1ffd98f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:23.694744 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc2a0b-10d5-4a65-9dfd-11a7d1ffd98f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:23.697212 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc2a0b-10d5-4a65-9dfd-11a7d1ffd98f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:23.709193 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 17/35] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 15:59:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:23.711771 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18a2a1-d491-40a7-a98d-60690f3a0fa6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:23.712416 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18a2a1-d491-40a7-a98d-60690f3a0fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:23.712910 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18a2a1-d491-40a7-a98d-60690f3a0fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:23.738376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-cb18a2a1-d491-40a7-a98d-60690f3a0fa6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:23.740248 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cb18a2a1-d491-40a7-a98d-60690f3a0fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:23.754226 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4100418-c15c-4332-8c14-56a1eb8586e1 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:23.800889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0b6cb-b85d-4ce0-8363-d55ad2d2a207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:23.801871 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0b6cb-b85d-4ce0-8363-d55ad2d2a207 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:23.802321 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0b6cb-b85d-4ce0-8363-d55ad2d2a207 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:23.802784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0b6cb-b85d-4ce0-8363-d55ad2d2a207 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:23.803863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0b6cb-b85d-4ce0-8363-d55ad2d2a207 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:23.804348 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0b6cb-b85d-4ce0-8363-d55ad2d2a207 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:23.806792 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0b6cb-b85d-4ce0-8363-d55ad2d2a207 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:23.821015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 18/36] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:23.821990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18a2a1-d491-40a7-a98d-60690f3a0fa6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:23.828332 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 19/37] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 15:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:23.830934 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc25b21-9998-4e0b-93c1-aad7540f5835 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:23.831653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc25b21-9998-4e0b-93c1-aad7540f5835 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:23.832357 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc25b21-9998-4e0b-93c1-aad7540f5835 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:23.835549 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 20/38] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:23.837632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc25b21-9998-4e0b-93c1-aad7540f5835 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:23.858726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:23.906310 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:23.986383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18a2a1-d491-40a7-a98d-60690f3a0fa6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:24.018310 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc25b21-9998-4e0b-93c1-aad7540f5835 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:24.050343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26aa9e-cf48-471c-b50a-aa291cc0c346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:24.051449 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26aa9e-cf48-471c-b50a-aa291cc0c346 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:24.051901 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26aa9e-cf48-471c-b50a-aa291cc0c346 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:24.052347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26aa9e-cf48-471c-b50a-aa291cc0c346 None admin] PATH_INFO: `/v3/users/feda116f177c415fa220633084d10d2f` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:24.054355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26aa9e-cf48-471c-b50a-aa291cc0c346 None admin] RBAC: Authorizing `identity:get_user(user_id=feda116f177c415fa220633084d10d2f)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:24.057736 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26aa9e-cf48-471c-b50a-aa291cc0c346 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:24.060399 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 19/39] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:23 2022] GET /identity/v3/users/feda116f177c415fa220633084d10d2f => generated 247 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:24.062371 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd29a8-c726-45bc-b89d-730ae2bc3d12 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:24.062843 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd29a8-c726-45bc-b89d-730ae2bc3d12 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:24.063288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd29a8-c726-45bc-b89d-730ae2bc3d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:24.091564 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054e614-7aad-4cea-9fb7-01419334bd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:24.092540 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054e614-7aad-4cea-9fb7-01419334bd5d None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:24.092694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-bacd29a8-c726-45bc-b89d-730ae2bc3d12 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:24.092981 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054e614-7aad-4cea-9fb7-01419334bd5d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:24.093428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054e614-7aad-4cea-9fb7-01419334bd5d None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:24.094589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bacd29a8-c726-45bc-b89d-730ae2bc3d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:24.095091 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054e614-7aad-4cea-9fb7-01419334bd5d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:24.097663 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054e614-7aad-4cea-9fb7-01419334bd5d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:24.154705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 21/40] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 6 15:59:23 2022] POST /identity/v3/projects => generated 301 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 15:59:24.154705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054e614-7aad-4cea-9fb7-01419334bd5d None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:24.205390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 20/41] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:24.216487 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd29a8-c726-45bc-b89d-730ae2bc3d12 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:24.261002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:24.343955 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:24.360771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054e614-7aad-4cea-9fb7-01419334bd5d None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:24.419376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd29a8-c726-45bc-b89d-730ae2bc3d12 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:24.447778 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b5fe7-dace-4b7e-8277-0c09028de1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:24.448601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b5fe7-dace-4b7e-8277-0c09028de1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:24.448896 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b5fe7-dace-4b7e-8277-0c09028de1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:24.449184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b5fe7-dace-4b7e-8277-0c09028de1f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:24.450901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b5fe7-dace-4b7e-8277-0c09028de1f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:24.453393 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b5fe7-dace-4b7e-8277-0c09028de1f3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:24.455633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 22/42] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 15:59:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:24.461689 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b5fe7-dace-4b7e-8277-0c09028de1f3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:24.491209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492d6c2-eb12-4b6d-bb32-1d5c448aaffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:24.491976 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492d6c2-eb12-4b6d-bb32-1d5c448aaffd None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:24.492950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492d6c2-eb12-4b6d-bb32-1d5c448aaffd None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:24.493297 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492d6c2-eb12-4b6d-bb32-1d5c448aaffd None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:24.493994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492d6c2-eb12-4b6d-bb32-1d5c448aaffd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:24.495433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492d6c2-eb12-4b6d-bb32-1d5c448aaffd None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:24.514767 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 21/43] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Sep 6 15:59:24 2022] POST /identity/v3/roles => generated 223 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 15:59:24.515592 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492d6c2-eb12-4b6d-bb32-1d5c448aaffd None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:24.614571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b5fe7-dace-4b7e-8277-0c09028de1f3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:24.653430 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:24.666140 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d063537-c612-40af-aba2-e9f216d38de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:24.667104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d063537-c612-40af-aba2-e9f216d38de3 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:24.667504 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d063537-c612-40af-aba2-e9f216d38de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:24.667967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d063537-c612-40af-aba2-e9f216d38de3 None admin] PATH_INFO: `/v3/domains/default/users/feda116f177c415fa220633084d10d2f/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:24.671266 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d063537-c612-40af-aba2-e9f216d38de3 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=feda116f177c415fa220633084d10d2f, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:24.672951 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d063537-c612-40af-aba2-e9f216d38de3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:24.695960 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 23/44] 10.222.0.26 () {58 vars in 1361 bytes} [Tue Sep 6 15:59:24 2022] PUT /identity/v3/domains/default/users/feda116f177c415fa220633084d10d2f/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 15:59:24.785733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492d6c2-eb12-4b6d-bb32-1d5c448aaffd None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:24.803491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:24.836139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecedf7b-8b47-4ef1-a3bd-011846118d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:24.837156 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecedf7b-8b47-4ef1-a3bd-011846118d2f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:24.837156 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecedf7b-8b47-4ef1-a3bd-011846118d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:24.837384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecedf7b-8b47-4ef1-a3bd-011846118d2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:24.838539 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecedf7b-8b47-4ef1-a3bd-011846118d2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:24.840155 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecedf7b-8b47-4ef1-a3bd-011846118d2f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:24.841835 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 22/45] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 15:59:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:24.847966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d063537-c612-40af-aba2-e9f216d38de3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:25.069990 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d063537-c612-40af-aba2-e9f216d38de3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:25.105318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0bd76d-22af-44f3-92e3-03a85409c797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:25.105948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0bd76d-22af-44f3-92e3-03a85409c797 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:25.106236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0bd76d-22af-44f3-92e3-03a85409c797 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:25.106515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0bd76d-22af-44f3-92e3-03a85409c797 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:25.107221 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0bd76d-22af-44f3-92e3-03a85409c797 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:25.108730 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0bd76d-22af-44f3-92e3-03a85409c797 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:25.136815 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 24/46] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 6 15:59:24 2022] POST /identity/v3/projects => generated 312 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 15:59:26.766744 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e2368-bea6-4119-9019-f8d5451d9e8e None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:26.767284 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e2368-bea6-4119-9019-f8d5451d9e8e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:26.767284 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e2368-bea6-4119-9019-f8d5451d9e8e None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:26.768852 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 23/47] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:26.775925 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c66310-ebc6-444f-a181-31f34ab0da4a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:26.776529 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c66310-ebc6-444f-a181-31f34ab0da4a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:26.776529 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c66310-ebc6-444f-a181-31f34ab0da4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:26.796542 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-49c66310-ebc6-444f-a181-31f34ab0da4a None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:26.798088 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-49c66310-ebc6-444f-a181-31f34ab0da4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:26.838936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:26.878928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 25/48] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:27.104717 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7674e-93f2-48ef-9a0f-d17344661e4e None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:27.105002 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7674e-93f2-48ef-9a0f-d17344661e4e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:27.105279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7674e-93f2-48ef-9a0f-d17344661e4e None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:27.106623 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 24/49] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:27.112159 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b2b6c-1d46-4727-9e12-1f5076a711cd None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:27.112453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b2b6c-1d46-4727-9e12-1f5076a711cd None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:27.112728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b2b6c-1d46-4727-9e12-1f5076a711cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:27.131884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-740b2b6c-1d46-4727-9e12-1f5076a711cd None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:27.133634 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-740b2b6c-1d46-4727-9e12-1f5076a711cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:27.162458 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7674e-93f2-48ef-9a0f-d17344661e4e None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:27.215403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 26/50] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:27.283834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7674e-93f2-48ef-9a0f-d17344661e4e None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:27.322865 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6309e11-15e5-4677-86a3-31d5018843e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:27.323513 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6309e11-15e5-4677-86a3-31d5018843e8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:27.323803 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6309e11-15e5-4677-86a3-31d5018843e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:27.324082 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6309e11-15e5-4677-86a3-31d5018843e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:27.343083 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6309e11-15e5-4677-86a3-31d5018843e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:27.348175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-b6309e11-15e5-4677-86a3-31d5018843e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 15:59:27.348175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:27.348175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:27.348175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:27.348175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:27.348175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:27.348175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:27.348175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:27.348175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:27.348175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:27.348175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:27.348175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:27.348175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:27.348175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:27.348175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:27.348175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:27.349815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:27.349815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:27.349815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:27.349815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:27.349815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:27.349815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:27.349815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:27.349815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:27.349815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:27.349815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:27.349815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:27.349815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:27.349815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:27.349815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:27.349815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:27.349815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:27.351858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:27.351858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:27.351858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:27.351858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:27.351858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:27.351858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:27.351858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:27.351858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:27.351858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:27.351858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:27.351858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 15:59:27.351858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 15:59:27.351858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 25/51] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 15:59:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 15:59:27.356495 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b2b6c-1d46-4727-9e12-1f5076a711cd None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:27.465362 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:27.468800 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b2b6c-1d46-4727-9e12-1f5076a711cd None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:27.490713 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6309e11-15e5-4677-86a3-31d5018843e8 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:27.513398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41cc49-9d1d-47d9-b3fb-4708553dc423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:27.514018 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41cc49-9d1d-47d9-b3fb-4708553dc423 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:27.514298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41cc49-9d1d-47d9-b3fb-4708553dc423 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:27.514570 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41cc49-9d1d-47d9-b3fb-4708553dc423 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:27.515287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41cc49-9d1d-47d9-b3fb-4708553dc423 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:27.515593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41cc49-9d1d-47d9-b3fb-4708553dc423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:27.517016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41cc49-9d1d-47d9-b3fb-4708553dc423 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:27.541085 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 27/52] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 15:59:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:27.542730 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41cc49-9d1d-47d9-b3fb-4708553dc423 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:27.547948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:27.620594 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6309e11-15e5-4677-86a3-31d5018843e8 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:27.660171 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775611b-2ccb-4481-8cc1-56616d647df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:27.660834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775611b-2ccb-4481-8cc1-56616d647df3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:27.661114 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775611b-2ccb-4481-8cc1-56616d647df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:27.661387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775611b-2ccb-4481-8cc1-56616d647df3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:27.662628 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775611b-2ccb-4481-8cc1-56616d647df3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:27.664824 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775611b-2ccb-4481-8cc1-56616d647df3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:27.666312 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41cc49-9d1d-47d9-b3fb-4708553dc423 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:27.666792 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 26/53] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 15:59:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 15:59:27.672595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775611b-2ccb-4481-8cc1-56616d647df3 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:27.707066 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87c30b-0e18-4c65-9961-0da8158bb1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:27.707729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87c30b-0e18-4c65-9961-0da8158bb1ec None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:27.708023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87c30b-0e18-4c65-9961-0da8158bb1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:27.708295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87c30b-0e18-4c65-9961-0da8158bb1ec None admin] PATH_INFO: `/v3/users/feda116f177c415fa220633084d10d2f` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:27.709589 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87c30b-0e18-4c65-9961-0da8158bb1ec None admin] RBAC: Authorizing `identity:get_user(user_id=feda116f177c415fa220633084d10d2f)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:27.712784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87c30b-0e18-4c65-9961-0da8158bb1ec None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:27.714567 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 28/54] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:27 2022] GET /identity/v3/users/feda116f177c415fa220633084d10d2f => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 15:59:27.721007 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87c30b-0e18-4c65-9961-0da8158bb1ec None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:27.757113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:27.803761 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775611b-2ccb-4481-8cc1-56616d647df3 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:27.816090 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:27.838237 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87c30b-0e18-4c65-9961-0da8158bb1ec None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:27.842986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58124ec3-e40d-4640-8395-3faa2df59428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:27.843602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58124ec3-e40d-4640-8395-3faa2df59428 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:27.843891 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58124ec3-e40d-4640-8395-3faa2df59428 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:27.844168 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58124ec3-e40d-4640-8395-3faa2df59428 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:27.844827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58124ec3-e40d-4640-8395-3faa2df59428 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:27.846274 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58124ec3-e40d-4640-8395-3faa2df59428 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:27.874765 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f681b25-959e-424b-944a-c1596b4b3830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:27.875400 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f681b25-959e-424b-944a-c1596b4b3830 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:27.875681 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f681b25-959e-424b-944a-c1596b4b3830 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:27.875864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 27/55] 10.222.0.26 () {60 vars in 1140 bytes} [Tue Sep 6 15:59:27 2022] POST /identity/v3/projects => generated 298 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 15:59:27.875959 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f681b25-959e-424b-944a-c1596b4b3830 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:27.877059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f681b25-959e-424b-944a-c1596b4b3830 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:27.878508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f681b25-959e-424b-944a-c1596b4b3830 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:27.880171 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 29/56] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 15:59:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:27.886978 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58124ec3-e40d-4640-8395-3faa2df59428 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:27.998447 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58124ec3-e40d-4640-8395-3faa2df59428 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:28.036237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf866526-046e-4df8-a180-622912650d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:28.036855 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf866526-046e-4df8-a180-622912650d33 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:28.037139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf866526-046e-4df8-a180-622912650d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:28.037440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf866526-046e-4df8-a180-622912650d33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:28.038146 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf866526-046e-4df8-a180-622912650d33 None admin] RBAC: Adding query filter params (role.id=ba05677650e046a48d5f19440d48fbf2, scope.domain.id=default, user.id=feda116f177c415fa220633084d10d2f) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:28.038457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf866526-046e-4df8-a180-622912650d33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:28.040522 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf866526-046e-4df8-a180-622912650d33 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:28.047194 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 28/57] 10.222.0.26 () {56 vars in 1322 bytes} [Tue Sep 6 15:59:27 2022] GET /identity/v3/role_assignments?user.id=feda116f177c415fa220633084d10d2f&scope.domain.id=default&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 530 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:29.794966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0ff35-58fd-4686-bc1f-0e0fe3550c7f None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:29.795471 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0ff35-58fd-4686-bc1f-0e0fe3550c7f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:29.795580 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0ff35-58fd-4686-bc1f-0e0fe3550c7f None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:29.796870 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 30/58] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:29.802390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4f9b9-a771-4b5b-ae1b-79555634ecfa None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:29.802786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4f9b9-a771-4b5b-ae1b-79555634ecfa None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:29.802988 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4f9b9-a771-4b5b-ae1b-79555634ecfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:29.824172 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-52d4f9b9-a771-4b5b-ae1b-79555634ecfa None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:29.825378 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-52d4f9b9-a771-4b5b-ae1b-79555634ecfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:29.846264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:29.893710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 29/59] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:30.171802 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0ff35-58fd-4686-bc1f-0e0fe3550c7f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:30.284946 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:30.290543 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0ff35-58fd-4686-bc1f-0e0fe3550c7f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:30.329120 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68747491-43b8-43ff-8f4e-3fc57ed3c4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:30.329759 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68747491-43b8-43ff-8f4e-3fc57ed3c4c2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:30.330043 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68747491-43b8-43ff-8f4e-3fc57ed3c4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:30.330351 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68747491-43b8-43ff-8f4e-3fc57ed3c4c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:30.331464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68747491-43b8-43ff-8f4e-3fc57ed3c4c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:30.332904 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68747491-43b8-43ff-8f4e-3fc57ed3c4c2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:30.334492 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 31/60] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 15:59:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:30.340649 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4f9b9-a771-4b5b-ae1b-79555634ecfa None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:30.448336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4f9b9-a771-4b5b-ae1b-79555634ecfa None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:30.483330 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150b844-1fb4-4a3f-a70b-ee7aef6adc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:30.483944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150b844-1fb4-4a3f-a70b-ee7aef6adc27 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:30.484227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150b844-1fb4-4a3f-a70b-ee7aef6adc27 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:30.484503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150b844-1fb4-4a3f-a70b-ee7aef6adc27 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:30.485157 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150b844-1fb4-4a3f-a70b-ee7aef6adc27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:30.486609 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150b844-1fb4-4a3f-a70b-ee7aef6adc27 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:30.514645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 30/61] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 6 15:59:30 2022] POST /identity/v3/users => generated 275 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 15:59:34.772338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc7ace-2656-4f72-b99a-5776c414d3ab None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:34.772907 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc7ace-2656-4f72-b99a-5776c414d3ab None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:34.773473 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc7ace-2656-4f72-b99a-5776c414d3ab None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:34.775014 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 32/62] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:34.790418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e527c-c340-4e1d-bbc3-7368844b9326 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:34.791715 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e527c-c340-4e1d-bbc3-7368844b9326 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:34.792212 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e527c-c340-4e1d-bbc3-7368844b9326 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:34.793616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abc71e-2ef0-404f-a52b-52c65dbc6457 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:34.794248 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 31/63] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:34.794550 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abc71e-2ef0-404f-a52b-52c65dbc6457 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:34.795253 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abc71e-2ef0-404f-a52b-52c65dbc6457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:34.800530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc332156-dc74-41fd-8198-57378efd13fe None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:34.801115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc332156-dc74-41fd-8198-57378efd13fe None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:34.801616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc332156-dc74-41fd-8198-57378efd13fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:34.822091 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-cc332156-dc74-41fd-8198-57378efd13fe None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:34.823712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cc332156-dc74-41fd-8198-57378efd13fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:34.832297 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-58abc71e-2ef0-404f-a52b-52c65dbc6457 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:34.833261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:34.833692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-58abc71e-2ef0-404f-a52b-52c65dbc6457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:34.918866 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 32/64] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:34.981097 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 33/65] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:35.014366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f39e9-4b30-44df-a21a-bda08f13c123 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:35.015004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f39e9-4b30-44df-a21a-bda08f13c123 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:35.015483 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f39e9-4b30-44df-a21a-bda08f13c123 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:35.017373 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 33/66] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:35.023767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11256a37-e259-4d7d-8983-39127ebf8377 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:35.024135 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11256a37-e259-4d7d-8983-39127ebf8377 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:35.024617 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11256a37-e259-4d7d-8983-39127ebf8377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:35.065909 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-11256a37-e259-4d7d-8983-39127ebf8377 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:35.068000 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-11256a37-e259-4d7d-8983-39127ebf8377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:35.170609 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f3be9-6972-47ec-8a47-3316671365f0 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:35.171299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f3be9-6972-47ec-8a47-3316671365f0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:35.171778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f3be9-6972-47ec-8a47-3316671365f0 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:35.178843 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 34/67] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:35.192429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfda78-d3c4-4ae3-bcb3-c278ca83b745 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:35.193150 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfda78-d3c4-4ae3-bcb3-c278ca83b745 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:35.193834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfda78-d3c4-4ae3-bcb3-c278ca83b745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:35.213464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 34/68] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:35.220647 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8bfbf-5d25-47e6-9935-6b603d1dbd93 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:35.221001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8bfbf-5d25-47e6-9935-6b603d1dbd93 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:35.221286 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8bfbf-5d25-47e6-9935-6b603d1dbd93 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:35.222602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-aebfda78-d3c4-4ae3-bcb3-c278ca83b745 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:35.222753 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 35/69] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 15:59:35.224764 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-aebfda78-d3c4-4ae3-bcb3-c278ca83b745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:35.229228 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102410b3-8f15-4587-9f17-5c94824cb7ce None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:35.229228 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102410b3-8f15-4587-9f17-5c94824cb7ce None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:35.229508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102410b3-8f15-4587-9f17-5c94824cb7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:35.250172 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-102410b3-8f15-4587-9f17-5c94824cb7ce None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:35.251528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-102410b3-8f15-4587-9f17-5c94824cb7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:35.277883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:35.306710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 35/70] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 15:59:35.307699 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfda78-d3c4-4ae3-bcb3-c278ca83b745 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:35.332958 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 36/71] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:35.334172 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102410b3-8f15-4587-9f17-5c94824cb7ce None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:35.387107 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:35.430542 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfda78-d3c4-4ae3-bcb3-c278ca83b745 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:35.484936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102410b3-8f15-4587-9f17-5c94824cb7ce None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:35.499428 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef3bdf-c7de-490c-bc23-b33efaa592e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:35.500109 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef3bdf-c7de-490c-bc23-b33efaa592e4 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:35.500398 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef3bdf-c7de-490c-bc23-b33efaa592e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:35.500678 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef3bdf-c7de-490c-bc23-b33efaa592e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:35.535871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef3bdf-c7de-490c-bc23-b33efaa592e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:35.541940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-ceef3bdf-c7de-490c-bc23-b33efaa592e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 15:59:35.541940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:35.541940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:35.541940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:35.541940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:35.541940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:35.541940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:35.541940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:35.541940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:35.541940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:35.541940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:35.541940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:35.541940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:35.541940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:35.541940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:35.541940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:35.542879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:35.542879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:35.542879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:35.542879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:35.542879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:35.542879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:35.542879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:35.542879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:35.542879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:35.542879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:35.542879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:35.542879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:35.542879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:35.542879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:35.542879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:35.542879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:35.543655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:35.543655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:35.543655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:35.543655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:35.543655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:35.543655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:35.543655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:35.543655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:35.543655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:35.543655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:35.543655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 15:59:35.543655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 15:59:35.546099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 36/72] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 15:59:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:35.547349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef3bdf-c7de-490c-bc23-b33efaa592e4 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:35.569143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8a839-b065-493d-a3cb-0fb72e23ecd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:35.570424 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8a839-b065-493d-a3cb-0fb72e23ecd1 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:35.570762 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8a839-b065-493d-a3cb-0fb72e23ecd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:35.571091 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8a839-b065-493d-a3cb-0fb72e23ecd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:35.586282 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f8a839-b065-493d-a3cb-0fb72e23ecd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:35.593520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-76f8a839-b065-493d-a3cb-0fb72e23ecd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 15:59:35.593520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:35.593520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:35.593520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:35.593520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:35.593520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:35.593520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:35.593520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:35.593520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:35.593520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:35.593520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:35.593520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:35.593520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:35.593520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:35.593520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:35.593520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:35.594438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:35.594438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:35.594438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:35.594438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:35.594438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:35.594438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:35.594438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:35.594438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:35.594438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:35.594438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:35.594438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:35.594438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:35.594438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:35.594438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:35.594438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:35.594438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:35.595238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:35.595238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:35.595238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:35.595238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:35.595238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:35.595238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:35.595238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:35.595238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:35.595238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:35.595238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:35.595238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 15:59:35.595238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 15:59:35.595238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 37/73] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 15:59:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:35.595854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8a839-b065-493d-a3cb-0fb72e23ecd1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:35.647065 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:35.680249 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef3bdf-c7de-490c-bc23-b33efaa592e4 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:35.714857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:35.721307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc511399-e789-488a-b129-1366ebd271cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:35.721945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc511399-e789-488a-b129-1366ebd271cb None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:35.722353 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc511399-e789-488a-b129-1366ebd271cb None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:35.722774 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8a839-b065-493d-a3cb-0fb72e23ecd1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:35.722902 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc511399-e789-488a-b129-1366ebd271cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:35.738380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc511399-e789-488a-b129-1366ebd271cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:35.745296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-cc511399-e789-488a-b129-1366ebd271cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 15:59:35.745296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:35.745296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:35.745296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:35.745296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:35.745296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:35.745296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:35.745296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:35.745296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:35.745296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:35.745296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:35.745296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:35.745296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:35.745296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:35.745296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:35.745296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:35.747015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:35.747015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:35.747015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:35.747015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:35.747015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:35.747015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:35.747015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:35.747015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:35.747015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:35.747015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:35.747015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:35.747015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:35.747015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:35.747015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:35.747015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:35.747015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:35.748633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:35.748633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:35.748633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:35.748633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:35.748633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:35.748633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:35.748633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:35.748633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:35.748633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:35.748633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:35.748633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 15:59:35.748633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 15:59:35.748633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 37/74] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 15:59:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:35.750866 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc511399-e789-488a-b129-1366ebd271cb None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:35.770894 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bf673-9729-4489-b9b6-2ad3064bbfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:35.771527 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bf673-9729-4489-b9b6-2ad3064bbfd5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:35.771812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bf673-9729-4489-b9b6-2ad3064bbfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:35.772083 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bf673-9729-4489-b9b6-2ad3064bbfd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:35.772781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bf673-9729-4489-b9b6-2ad3064bbfd5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:35.773864 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bf673-9729-4489-b9b6-2ad3064bbfd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:35.774644 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bf673-9729-4489-b9b6-2ad3064bbfd5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:35.784679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 38/75] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 15:59:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:35.785576 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bf673-9729-4489-b9b6-2ad3064bbfd5 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:35.893218 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:35.894621 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc511399-e789-488a-b129-1366ebd271cb None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:35.918729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bf673-9729-4489-b9b6-2ad3064bbfd5 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:35.943876 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5bf6a-ea18-4430-89b6-3231be8659da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:35.945405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5bf6a-ea18-4430-89b6-3231be8659da None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:35.945899 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5bf6a-ea18-4430-89b6-3231be8659da None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:35.946369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5bf6a-ea18-4430-89b6-3231be8659da None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:35.947589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5bf6a-ea18-4430-89b6-3231be8659da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:35.948108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5bf6a-ea18-4430-89b6-3231be8659da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:35.950820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5bf6a-ea18-4430-89b6-3231be8659da None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:35.957337 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a4c95-0467-48d0-a1b2-62e1cf58338a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:35.957982 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a4c95-0467-48d0-a1b2-62e1cf58338a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:35.958268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a4c95-0467-48d0-a1b2-62e1cf58338a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:35.958546 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a4c95-0467-48d0-a1b2-62e1cf58338a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:35.971644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 38/76] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 15:59:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:35.971644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5bf6a-ea18-4430-89b6-3231be8659da None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:35.971893 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a4c95-0467-48d0-a1b2-62e1cf58338a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:35.977115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-246a4c95-0467-48d0-a1b2-62e1cf58338a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 15:59:35.977115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:35.977115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:35.977115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:35.977115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:35.977115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:35.977115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:35.977115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:35.977115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:35.977115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:35.977115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:35.977115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:35.977115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:35.977115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:35.977115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:35.977115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:35.978303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:35.978303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:35.978303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:35.978303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:35.978303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:35.978303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:35.978303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:35.978303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:35.978303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:35.978303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:35.978303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:35.978303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:35.978303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:35.978303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:35.978303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:35.978303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:35.983997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:35.983997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:35.983997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:35.983997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:35.983997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:35.983997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:35.983997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:35.983997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:35.983997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:35.983997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:35.983997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 15:59:35.983997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 15:59:35.983997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 39/77] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 15:59:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:35.983997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a4c95-0467-48d0-a1b2-62e1cf58338a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:36.067998 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:36.095957 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a4c95-0467-48d0-a1b2-62e1cf58338a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:36.141414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5bf6a-ea18-4430-89b6-3231be8659da None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:36.149791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc116a9f-17d5-43bd-991c-f2ee273e0d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:36.150759 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc116a9f-17d5-43bd-991c-f2ee273e0d89 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:36.151207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc116a9f-17d5-43bd-991c-f2ee273e0d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:36.151649 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc116a9f-17d5-43bd-991c-f2ee273e0d89 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:36.152752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc116a9f-17d5-43bd-991c-f2ee273e0d89 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:36.153253 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc116a9f-17d5-43bd-991c-f2ee273e0d89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:36.155473 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc116a9f-17d5-43bd-991c-f2ee273e0d89 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:36.163281 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:36.168739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 40/78] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 15:59:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:36.169708 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc116a9f-17d5-43bd-991c-f2ee273e0d89 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:36.204047 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fd0b3-37cc-4362-8521-bb765bf513c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:36.205620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fd0b3-37cc-4362-8521-bb765bf513c9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:36.206120 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fd0b3-37cc-4362-8521-bb765bf513c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:36.206622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fd0b3-37cc-4362-8521-bb765bf513c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:36.208459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fd0b3-37cc-4362-8521-bb765bf513c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:36.212885 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fd0b3-37cc-4362-8521-bb765bf513c9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:36.214923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 39/79] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 15:59:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:36.216260 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fd0b3-37cc-4362-8521-bb765bf513c9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:36.307140 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc116a9f-17d5-43bd-991c-f2ee273e0d89 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:36.341543 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fd0b3-37cc-4362-8521-bb765bf513c9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:36.352935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6e4db-e98c-40c4-83c8-28aefe76fb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:36.353670 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6e4db-e98c-40c4-83c8-28aefe76fb71 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:36.354025 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6e4db-e98c-40c4-83c8-28aefe76fb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:36.354482 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6e4db-e98c-40c4-83c8-28aefe76fb71 None admin] PATH_INFO: `/v3/users/5bbe86494fef439facc3d7e23e115ded` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:36.371031 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:36.375853 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6e4db-e98c-40c4-83c8-28aefe76fb71 None admin] RBAC: Authorizing `identity:get_user(user_id=5bbe86494fef439facc3d7e23e115ded)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:36.377633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6e4db-e98c-40c4-83c8-28aefe76fb71 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:36.379410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 41/80] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:36 2022] GET /identity/v3/users/5bbe86494fef439facc3d7e23e115ded => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:36.380201 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6e4db-e98c-40c4-83c8-28aefe76fb71 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:36.385954 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe7389-e908-4c3a-ade8-661ad44e6647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:36.387566 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe7389-e908-4c3a-ade8-661ad44e6647 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:36.387855 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe7389-e908-4c3a-ade8-661ad44e6647 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:36.388132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe7389-e908-4c3a-ade8-661ad44e6647 None admin] PATH_INFO: `/v3/users/feda116f177c415fa220633084d10d2f` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:36.389346 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe7389-e908-4c3a-ade8-661ad44e6647 None admin] RBAC: Authorizing `identity:get_user(user_id=feda116f177c415fa220633084d10d2f)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:36.390871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe7389-e908-4c3a-ade8-661ad44e6647 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:36.392524 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 40/81] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:36 2022] GET /identity/v3/users/feda116f177c415fa220633084d10d2f => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:36.393315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe7389-e908-4c3a-ade8-661ad44e6647 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:36.483422 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:36.512323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe7389-e908-4c3a-ade8-661ad44e6647 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:36.546002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6e4db-e98c-40c4-83c8-28aefe76fb71 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:36.552464 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4535c-eb40-4e86-8fc5-58291639cf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:36.553202 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4535c-eb40-4e86-8fc5-58291639cf91 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:36.553564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4535c-eb40-4e86-8fc5-58291639cf91 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:36.553937 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4535c-eb40-4e86-8fc5-58291639cf91 None admin] PATH_INFO: `/v3/users/5bbe86494fef439facc3d7e23e115ded` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:36.555314 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe4535c-eb40-4e86-8fc5-58291639cf91 None admin] RBAC: Authorizing `identity:get_user(user_id=5bbe86494fef439facc3d7e23e115ded)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:36.556873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe4535c-eb40-4e86-8fc5-58291639cf91 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:36.566785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 41/82] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:36 2022] GET /identity/v3/users/5bbe86494fef439facc3d7e23e115ded => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:36.566785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4535c-eb40-4e86-8fc5-58291639cf91 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:36.611121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee7ddd-7231-479d-b8e7-8161377b7716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:36.611828 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee7ddd-7231-479d-b8e7-8161377b7716 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:36.612113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee7ddd-7231-479d-b8e7-8161377b7716 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:36.612386 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee7ddd-7231-479d-b8e7-8161377b7716 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:36.613066 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee7ddd-7231-479d-b8e7-8161377b7716 None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:36.613371 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee7ddd-7231-479d-b8e7-8161377b7716 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:36.614794 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee7ddd-7231-479d-b8e7-8161377b7716 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:36.624525 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 42/83] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 15:59:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:36.625818 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee7ddd-7231-479d-b8e7-8161377b7716 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:36.633940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:36.702476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4535c-eb40-4e86-8fc5-58291639cf91 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:36.754487 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c76420-1874-4733-a5c6-a978a6db91f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:36.755564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c76420-1874-4733-a5c6-a978a6db91f6 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:36.756085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c76420-1874-4733-a5c6-a978a6db91f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:36.756591 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c76420-1874-4733-a5c6-a978a6db91f6 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:36.757793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c76420-1874-4733-a5c6-a978a6db91f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:36.760751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c76420-1874-4733-a5c6-a978a6db91f6 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:36.769556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee7ddd-7231-479d-b8e7-8161377b7716 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:36.794088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 42/84] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 6 15:59:36 2022] POST /identity/v3/users => generated 289 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 15:59:36.807205 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c76420-1874-4733-a5c6-a978a6db91f6 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:36.807365 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:36.815502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceae55e-2cde-4d40-bd21-0215111ac37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:36.816263 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceae55e-2cde-4d40-bd21-0215111ac37c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:36.816557 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceae55e-2cde-4d40-bd21-0215111ac37c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:36.817039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceae55e-2cde-4d40-bd21-0215111ac37c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:36.907823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceae55e-2cde-4d40-bd21-0215111ac37c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:36.909520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceae55e-2cde-4d40-bd21-0215111ac37c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:36.959893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c76420-1874-4733-a5c6-a978a6db91f6 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:36.979904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:37.008680 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e2978-8a8e-42d0-a615-f6555e1eea29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:37.009312 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e2978-8a8e-42d0-a615-f6555e1eea29 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:37.009590 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e2978-8a8e-42d0-a615-f6555e1eea29 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:37.009928 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e2978-8a8e-42d0-a615-f6555e1eea29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:37.049946 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 43/85] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 15:59:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 15:59:37.050999 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceae55e-2cde-4d40-bd21-0215111ac37c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:37.090163 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:37.124921 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e2978-8a8e-42d0-a615-f6555e1eea29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:37.127480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e2978-8a8e-42d0-a615-f6555e1eea29 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:37.268666 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceae55e-2cde-4d40-bd21-0215111ac37c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:37.336727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 43/86] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 15:59:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 15:59:37.338428 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e2978-8a8e-42d0-a615-f6555e1eea29 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:37.341223 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95bd87-6c05-4680-8147-b73df21e7eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:37.341933 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95bd87-6c05-4680-8147-b73df21e7eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:37.342339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95bd87-6c05-4680-8147-b73df21e7eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:37.342812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95bd87-6c05-4680-8147-b73df21e7eb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:37.384017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:37.475898 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95bd87-6c05-4680-8147-b73df21e7eb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:37.479901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95bd87-6c05-4680-8147-b73df21e7eb6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:37.523442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:37.594474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e2978-8a8e-42d0-a615-f6555e1eea29 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:37.681817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520d656-c204-490c-8078-d4ac4e312600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:37.684174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520d656-c204-490c-8078-d4ac4e312600 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:37.685043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520d656-c204-490c-8078-d4ac4e312600 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:37.686166 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520d656-c204-490c-8078-d4ac4e312600 None admin] PATH_INFO: `/v3/users/5bbe86494fef439facc3d7e23e115ded` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:37.688922 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520d656-c204-490c-8078-d4ac4e312600 None admin] RBAC: Authorizing `identity:get_user(user_id=5bbe86494fef439facc3d7e23e115ded)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:37.692976 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520d656-c204-490c-8078-d4ac4e312600 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:37.698803 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 44/87] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:37 2022] GET /identity/v3/users/5bbe86494fef439facc3d7e23e115ded => generated 275 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:37.701044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520d656-c204-490c-8078-d4ac4e312600 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:37.719562 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 44/88] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 15:59:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 669 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 15:59:37.722694 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95bd87-6c05-4680-8147-b73df21e7eb6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:37.884482 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:37.924133 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520d656-c204-490c-8078-d4ac4e312600 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:37.957021 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95bd87-6c05-4680-8147-b73df21e7eb6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:37.995569 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae59a42-3aad-4949-886d-596e66f8d4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:37.996733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae59a42-3aad-4949-886d-596e66f8d4da None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:37.997256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae59a42-3aad-4949-886d-596e66f8d4da None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:37.997766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae59a42-3aad-4949-886d-596e66f8d4da None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:38.000203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae59a42-3aad-4949-886d-596e66f8d4da None admin] RBAC: Authorizing `identity:get_project(project_id=dda7941c8f444d41afc018b93fcd36fe)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:38.003075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:38.003287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae59a42-3aad-4949-886d-596e66f8d4da None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:38.007050 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 45/89] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 15:59:37 2022] GET /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe => generated 298 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:38.009463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae59a42-3aad-4949-886d-596e66f8d4da None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:38.027472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2f110-72d1-47bd-bb64-5858cfbfa8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:38.028133 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2f110-72d1-47bd-bb64-5858cfbfa8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:38.028415 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2f110-72d1-47bd-bb64-5858cfbfa8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:38.028685 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2f110-72d1-47bd-bb64-5858cfbfa8a4 None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:38.029957 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2f110-72d1-47bd-bb64-5858cfbfa8a4 None admin] RBAC: Authorizing `identity:get_project(project_id=dda7941c8f444d41afc018b93fcd36fe)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:38.031515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2f110-72d1-47bd-bb64-5858cfbfa8a4 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:38.038246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 45/90] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 15:59:37 2022] GET /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe => generated 298 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:38.040127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2f110-72d1-47bd-bb64-5858cfbfa8a4 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:38.155880 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2f110-72d1-47bd-bb64-5858cfbfa8a4 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:38.198680 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08e830-30df-4ea8-9ae7-a9c206d7e137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:38.199520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08e830-30df-4ea8-9ae7-a9c206d7e137 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:38.199809 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08e830-30df-4ea8-9ae7-a9c206d7e137 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:38.200095 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08e830-30df-4ea8-9ae7-a9c206d7e137 None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:38.201300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08e830-30df-4ea8-9ae7-a9c206d7e137 None admin] RBAC: Authorizing `identity:get_project(project_id=dda7941c8f444d41afc018b93fcd36fe)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:38.202810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08e830-30df-4ea8-9ae7-a9c206d7e137 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:38.204834 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 46/91] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 15:59:38 2022] GET /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:38.205649 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08e830-30df-4ea8-9ae7-a9c206d7e137 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:38.211733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae59a42-3aad-4949-886d-596e66f8d4da None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:38.271317 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:38.281858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411e1c8-9721-48bb-9418-a03ecafd4661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:38.283001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411e1c8-9721-48bb-9418-a03ecafd4661 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:38.283653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411e1c8-9721-48bb-9418-a03ecafd4661 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:38.284151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411e1c8-9721-48bb-9418-a03ecafd4661 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:38.322385 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08e830-30df-4ea8-9ae7-a9c206d7e137 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:38.324793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:38.360652 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a96bd-a094-4d69-a2be-cc765c41b710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:38.361266 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a96bd-a094-4d69-a2be-cc765c41b710 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:38.361541 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a96bd-a094-4d69-a2be-cc765c41b710 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:38.361813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a96bd-a094-4d69-a2be-cc765c41b710 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:38.362573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a96bd-a094-4d69-a2be-cc765c41b710 None admin] RBAC: Adding query filter params (role.id=e89162686187404da84651e0a3632b7b, scope.project.id=dda7941c8f444d41afc018b93fcd36fe, user.id=5bbe86494fef439facc3d7e23e115ded) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:38.364068 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a96bd-a094-4d69-a2be-cc765c41b710 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:38.364363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a96bd-a094-4d69-a2be-cc765c41b710 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:38.378268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 47/92] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 15:59:38 2022] GET /identity/v3/role_assignments?user.id=5bbe86494fef439facc3d7e23e115ded&scope.project.id=dda7941c8f444d41afc018b93fcd36fe&role.id=e89162686187404da84651e0a3632b7b => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:38.379324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a96bd-a094-4d69-a2be-cc765c41b710 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:38.433860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411e1c8-9721-48bb-9418-a03ecafd4661 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:38.437431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411e1c8-9721-48bb-9418-a03ecafd4661 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:38.574928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a96bd-a094-4d69-a2be-cc765c41b710 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:38.607412 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:38.620868 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14635f-d944-41a9-83e1-ba5c08a8099a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:38.621662 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14635f-d944-41a9-83e1-ba5c08a8099a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:38.621812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14635f-d944-41a9-83e1-ba5c08a8099a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:38.622084 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14635f-d944-41a9-83e1-ba5c08a8099a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:38.622880 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14635f-d944-41a9-83e1-ba5c08a8099a None admin] RBAC: Adding query filter params (role.id=ba05677650e046a48d5f19440d48fbf2, scope.project.id=dda7941c8f444d41afc018b93fcd36fe, user.id=feda116f177c415fa220633084d10d2f) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:38.623190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14635f-d944-41a9-83e1-ba5c08a8099a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:38.630739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14635f-d944-41a9-83e1-ba5c08a8099a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:38.631029 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 48/93] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 15:59:38 2022] GET /identity/v3/role_assignments?user.id=feda116f177c415fa220633084d10d2f&scope.project.id=dda7941c8f444d41afc018b93fcd36fe&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:38.632128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14635f-d944-41a9-83e1-ba5c08a8099a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:38.727873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 46/94] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 15:59:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 15:59:38.736547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411e1c8-9721-48bb-9418-a03ecafd4661 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:38.867273 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:38.879391 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14635f-d944-41a9-83e1-ba5c08a8099a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:38.955177 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decf523-643d-4957-84d8-e459c82af278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:38.955811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decf523-643d-4957-84d8-e459c82af278 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:38.956097 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decf523-643d-4957-84d8-e459c82af278 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:38.956372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decf523-643d-4957-84d8-e459c82af278 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:38.957124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decf523-643d-4957-84d8-e459c82af278 None admin] RBAC: Adding query filter params (role.id=bad9ff35005240e99bedef3fddb1dc77, scope.project.id=dda7941c8f444d41afc018b93fcd36fe, user.id=5bbe86494fef439facc3d7e23e115ded) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:38.957662 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decf523-643d-4957-84d8-e459c82af278 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:38.958924 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decf523-643d-4957-84d8-e459c82af278 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:38.982450 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 49/95] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 15:59:38 2022] GET /identity/v3/role_assignments?user.id=5bbe86494fef439facc3d7e23e115ded&scope.project.id=dda7941c8f444d41afc018b93fcd36fe&role.id=bad9ff35005240e99bedef3fddb1dc77 => generated 259 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:39.002776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411e1c8-9721-48bb-9418-a03ecafd4661 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:39.079503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918d79b-b26f-461b-a46d-150613c57e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:39.080647 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918d79b-b26f-461b-a46d-150613c57e27 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:39.081138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918d79b-b26f-461b-a46d-150613c57e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:39.081604 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918d79b-b26f-461b-a46d-150613c57e27 None admin] PATH_INFO: `/v3/projects/36229525382149a3831f7f4dc0b0e575` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:39.085793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918d79b-b26f-461b-a46d-150613c57e27 None admin] RBAC: Authorizing `identity:get_project(project_id=36229525382149a3831f7f4dc0b0e575)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:39.088931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918d79b-b26f-461b-a46d-150613c57e27 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:39.092256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 47/96] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 15:59:38 2022] GET /identity/v3/projects/36229525382149a3831f7f4dc0b0e575 => generated 312 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:39.099534 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decf523-643d-4957-84d8-e459c82af278 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:39.226019 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:39.294392 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decf523-643d-4957-84d8-e459c82af278 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:39.337088 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8853ff-c5ed-4c6d-bc2d-e93f73b8249a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:39.337723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8853ff-c5ed-4c6d-bc2d-e93f73b8249a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:39.338005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8853ff-c5ed-4c6d-bc2d-e93f73b8249a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:39.338287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8853ff-c5ed-4c6d-bc2d-e93f73b8249a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:39.339107 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8853ff-c5ed-4c6d-bc2d-e93f73b8249a None admin] RBAC: Adding query filter params (role.id=e89162686187404da84651e0a3632b7b, scope.project.id=36229525382149a3831f7f4dc0b0e575, user.id=5bbe86494fef439facc3d7e23e115ded) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:39.339421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8853ff-c5ed-4c6d-bc2d-e93f73b8249a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:39.340948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8853ff-c5ed-4c6d-bc2d-e93f73b8249a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:39.347576 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 50/97] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 15:59:39 2022] GET /identity/v3/role_assignments?user.id=5bbe86494fef439facc3d7e23e115ded&scope.project.id=36229525382149a3831f7f4dc0b0e575&role.id=e89162686187404da84651e0a3632b7b => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:39.714437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdf380-14b1-4a96-bcf8-a8d18ce95874 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:39.714963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdf380-14b1-4a96-bcf8-a8d18ce95874 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:39.715459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdf380-14b1-4a96-bcf8-a8d18ce95874 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:39.718012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 48/98] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:39.725149 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e19b87-2e70-4afb-876f-0b0ddf4fae28 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:39.725650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e19b87-2e70-4afb-876f-0b0ddf4fae28 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:39.726680 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e19b87-2e70-4afb-876f-0b0ddf4fae28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:39.766885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a0e19b87-2e70-4afb-876f-0b0ddf4fae28 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:39.769070 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a0e19b87-2e70-4afb-876f-0b0ddf4fae28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:39.900368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 51/99] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:40.343823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86af55a-c420-4a5a-b6bb-9291cde17330 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:40.344337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86af55a-c420-4a5a-b6bb-9291cde17330 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:40.344776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86af55a-c420-4a5a-b6bb-9291cde17330 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:40.347283 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 49/100] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:40.357836 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f112a31-dc24-4c13-a82e-ee99f12bd696 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:40.358335 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f112a31-dc24-4c13-a82e-ee99f12bd696 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:40.358866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f112a31-dc24-4c13-a82e-ee99f12bd696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:40.401067 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-5f112a31-dc24-4c13-a82e-ee99f12bd696 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:40.402937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5f112a31-dc24-4c13-a82e-ee99f12bd696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:40.485652 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:40.535278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 52/101] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:40.793927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86af55a-c420-4a5a-b6bb-9291cde17330 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:41.019935 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86af55a-c420-4a5a-b6bb-9291cde17330 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:41.056115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:41.095708 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e257d4e1-d96a-4b32-852e-c4a07205782b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:41.096780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257d4e1-d96a-4b32-852e-c4a07205782b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:41.097276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257d4e1-d96a-4b32-852e-c4a07205782b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:41.097760 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257d4e1-d96a-4b32-852e-c4a07205782b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:41.118295 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e257d4e1-d96a-4b32-852e-c4a07205782b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:41.128385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-e257d4e1-d96a-4b32-852e-c4a07205782b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 15:59:41.128385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:41.128385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:41.128385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:41.128385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:41.128385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:41.128385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:41.128385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:41.128385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:41.128385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:41.128385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:41.128385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:41.128385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:41.128385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:41.128385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:41.128385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:41.129390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:41.129390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:41.129390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:41.129390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:41.129390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:41.129390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:41.129390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:41.129390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:41.129390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:41.129390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:41.129390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:41.129390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:41.129390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:41.129390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:41.129390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:41.129390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:41.130132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:41.130132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:41.130132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:41.130132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:41.130132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:41.130132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:41.130132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:41.130132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:41.130132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:41.130132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:41.130132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 15:59:41.130132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 15:59:41.130722 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 50/102] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 15:59:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:41.142015 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f112a31-dc24-4c13-a82e-ee99f12bd696 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:41.285645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e257d4e1-d96a-4b32-852e-c4a07205782b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:41.380925 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f112a31-dc24-4c13-a82e-ee99f12bd696 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:41.498443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e257d4e1-d96a-4b32-852e-c4a07205782b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:41.544789 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b53f2-4526-49e7-977a-bc40d10dea4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:41.545718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b53f2-4526-49e7-977a-bc40d10dea4e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:41.546216 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b53f2-4526-49e7-977a-bc40d10dea4e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:41.546721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b53f2-4526-49e7-977a-bc40d10dea4e None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:41.547919 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b53f2-4526-49e7-977a-bc40d10dea4e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:41.548467 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b53f2-4526-49e7-977a-bc40d10dea4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:41.551352 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b53f2-4526-49e7-977a-bc40d10dea4e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:41.573421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 53/103] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 15:59:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:41.576869 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0161302-7010-4ec7-a5a5-bb726c56b4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:41.578203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0161302-7010-4ec7-a5a5-bb726c56b4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:41.578968 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0161302-7010-4ec7-a5a5-bb726c56b4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:41.579662 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b53f2-4526-49e7-977a-bc40d10dea4e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:41.579969 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0161302-7010-4ec7-a5a5-bb726c56b4d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:41.582231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0161302-7010-4ec7-a5a5-bb726c56b4d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:41.586058 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0161302-7010-4ec7-a5a5-bb726c56b4d3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:41.590750 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 51/104] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 15:59:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:41.611512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0161302-7010-4ec7-a5a5-bb726c56b4d3 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:41.665989 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:41.719970 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:41.829884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0161302-7010-4ec7-a5a5-bb726c56b4d3 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:41.893115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b53f2-4526-49e7-977a-bc40d10dea4e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:41.903464 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9184e-f895-4bfe-8124-2f0783c14e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:41.905004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9184e-f895-4bfe-8124-2f0783c14e3f None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:41.905772 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9184e-f895-4bfe-8124-2f0783c14e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:41.906534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9184e-f895-4bfe-8124-2f0783c14e3f None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:41.908149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d9184e-f895-4bfe-8124-2f0783c14e3f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:41.911162 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d9184e-f895-4bfe-8124-2f0783c14e3f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:41.955817 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f9a74-7c21-44f8-9a3d-0dba183061bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:41.957165 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f9a74-7c21-44f8-9a3d-0dba183061bd None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:41.957945 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f9a74-7c21-44f8-9a3d-0dba183061bd None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:41.958654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f9a74-7c21-44f8-9a3d-0dba183061bd None admin] PATH_INFO: `/v3/users/01f72ea34d5f4ee5b72408d14e925ffe` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:41.985782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f9a74-7c21-44f8-9a3d-0dba183061bd None admin] RBAC: Authorizing `identity:get_user(user_id=01f72ea34d5f4ee5b72408d14e925ffe)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:41.989170 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f9a74-7c21-44f8-9a3d-0dba183061bd None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:41.993678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 54/105] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:41 2022] GET /identity/v3/users/01f72ea34d5f4ee5b72408d14e925ffe => generated 289 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 15:59:41.996265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536957c-86ae-4944-a3d4-9e5e758e6c8b None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:41.997072 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536957c-86ae-4944-a3d4-9e5e758e6c8b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:41.997805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536957c-86ae-4944-a3d4-9e5e758e6c8b None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:42.000965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 55/106] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:42.002569 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536957c-86ae-4944-a3d4-9e5e758e6c8b None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:42.005410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 52/107] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 6 15:59:41 2022] POST /identity/v3/projects => generated 302 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 15:59:42.009466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d51eb5-28aa-4ad8-bdcb-8d539152c20f None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:42.010261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d51eb5-28aa-4ad8-bdcb-8d539152c20f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:42.014692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d51eb5-28aa-4ad8-bdcb-8d539152c20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:42.080633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d6d51eb5-28aa-4ad8-bdcb-8d539152c20f None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:42.083016 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d6d51eb5-28aa-4ad8-bdcb-8d539152c20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:42.201895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:42.209332 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536957c-86ae-4944-a3d4-9e5e758e6c8b None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:42.274703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209f11d-e109-4843-bab5-46ee675bbf33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:42.274703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209f11d-e109-4843-bab5-46ee675bbf33 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:42.275448 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209f11d-e109-4843-bab5-46ee675bbf33 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:42.276012 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209f11d-e109-4843-bab5-46ee675bbf33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:42.294758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 53/108] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:42.340602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078794d4-67a1-48a0-af5f-65da139a163e None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:42.341521 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078794d4-67a1-48a0-af5f-65da139a163e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:42.342285 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078794d4-67a1-48a0-af5f-65da139a163e None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:42.349303 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 54/109] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:42 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 15:59:42.357117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405184a-24f1-4d56-bac4-5ad745953bd6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:42.358048 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405184a-24f1-4d56-bac4-5ad745953bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:42.358857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405184a-24f1-4d56-bac4-5ad745953bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:42.416699 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-3405184a-24f1-4d56-bac4-5ad745953bd6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:42.419157 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3405184a-24f1-4d56-bac4-5ad745953bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:42.420176 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209f11d-e109-4843-bab5-46ee675bbf33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:42.423767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209f11d-e109-4843-bab5-46ee675bbf33 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:42.507672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:42.626368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:42.648895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 56/110] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 15:59:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 15:59:42.658477 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209f11d-e109-4843-bab5-46ee675bbf33 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:42.689087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 55/111] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:42.757265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405184a-24f1-4d56-bac4-5ad745953bd6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:42.844441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209f11d-e109-4843-bab5-46ee675bbf33 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:42.894866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9066b-9019-4819-a505-b569cf7b4dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:42.895778 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9066b-9019-4819-a505-b569cf7b4dbf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:42.896220 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9066b-9019-4819-a505-b569cf7b4dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:42.896671 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9066b-9019-4819-a505-b569cf7b4dbf None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:42.903656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9066b-9019-4819-a505-b569cf7b4dbf None admin] RBAC: Authorizing `identity:get_project(project_id=dda7941c8f444d41afc018b93fcd36fe)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:42.905554 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9066b-9019-4819-a505-b569cf7b4dbf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:42.908018 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 57/112] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 15:59:42 2022] GET /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:42.909492 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ee350-39b2-4b8c-9144-925a28748976 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:42.910007 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ee350-39b2-4b8c-9144-925a28748976 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:42.910430 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ee350-39b2-4b8c-9144-925a28748976 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:42.912222 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 58/113] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:42.918944 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ee350-39b2-4b8c-9144-925a28748976 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:42.930005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405184a-24f1-4d56-bac4-5ad745953bd6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:42.990544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8e463-73f9-4537-8e74-f67e9e4b3305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:42.992027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8e463-73f9-4537-8e74-f67e9e4b3305 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:42.992461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8e463-73f9-4537-8e74-f67e9e4b3305 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:42.992944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8e463-73f9-4537-8e74-f67e9e4b3305 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:43.008821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:43.013730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8e463-73f9-4537-8e74-f67e9e4b3305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:43.027456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-2fc8e463-73f9-4537-8e74-f67e9e4b3305 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 15:59:43.027456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:43.027456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:43.027456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:43.027456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:43.027456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:43.027456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:43.027456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:43.027456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:43.027456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:43.027456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:43.027456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:43.027456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:43.027456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:43.027456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:43.027456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:43.028477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:43.028477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:43.028477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:43.028477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:43.028477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:43.028477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:43.028477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:43.028477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:43.028477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:43.028477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:43.028477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:43.028477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:43.028477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:43.028477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:43.028477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:43.028477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:43.029243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:43.029243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:43.029243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:43.029243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:43.029243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:43.029243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:43.029243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:43.029243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:43.029243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:43.029243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:43.029243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 15:59:43.029243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 15:59:43.029243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 56/114] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 15:59:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 15:59:43.030619 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539d122-b7ab-4bb1-8ea1-fcdad6993bca None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:43.034888 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539d122-b7ab-4bb1-8ea1-fcdad6993bca None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:43.034888 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539d122-b7ab-4bb1-8ea1-fcdad6993bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:43.068671 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0539d122-b7ab-4bb1-8ea1-fcdad6993bca None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:43.070862 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0539d122-b7ab-4bb1-8ea1-fcdad6993bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:43.080771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ee350-39b2-4b8c-9144-925a28748976 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:43.106153 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:43.157894 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f972ba9-c793-4d6a-a3b8-be6450823be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:43.158471 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f972ba9-c793-4d6a-a3b8-be6450823be8 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:43.158787 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f972ba9-c793-4d6a-a3b8-be6450823be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:43.159064 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f972ba9-c793-4d6a-a3b8-be6450823be8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:43.161460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f972ba9-c793-4d6a-a3b8-be6450823be8 None admin] RBAC: Adding query filter params (role.id=11b32861fcd8469382d1a7047a95ea70, scope.project.id=dda7941c8f444d41afc018b93fcd36fe, user.id=01f72ea34d5f4ee5b72408d14e925ffe) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:43.161783 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f972ba9-c793-4d6a-a3b8-be6450823be8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:43.163953 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f972ba9-c793-4d6a-a3b8-be6450823be8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:43.171233 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 59/115] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 15:59:42 2022] GET /identity/v3/role_assignments?user.id=01f72ea34d5f4ee5b72408d14e925ffe&scope.project.id=dda7941c8f444d41afc018b93fcd36fe&role.id=11b32861fcd8469382d1a7047a95ea70 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 15:59:43.172499 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f972ba9-c793-4d6a-a3b8-be6450823be8 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:43.289067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 57/116] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:43.290682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539d122-b7ab-4bb1-8ea1-fcdad6993bca None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:43.396380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f972ba9-c793-4d6a-a3b8-be6450823be8 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:43.461664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:43.488725 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7316280-182a-4436-be23-d7c8233dee83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:43.489353 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7316280-182a-4436-be23-d7c8233dee83 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:43.489631 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7316280-182a-4436-be23-d7c8233dee83 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:43.489907 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7316280-182a-4436-be23-d7c8233dee83 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:43.490594 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7316280-182a-4436-be23-d7c8233dee83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:43.490927 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7316280-182a-4436-be23-d7c8233dee83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:43.494739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7316280-182a-4436-be23-d7c8233dee83 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:43.515965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 60/117] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 15:59:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:43.520426 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539d122-b7ab-4bb1-8ea1-fcdad6993bca None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:43.524967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7316280-182a-4436-be23-d7c8233dee83 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:43.619858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1f978-a7ee-4332-b3b8-177010ef21a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:43.620519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1f978-a7ee-4332-b3b8-177010ef21a9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:43.620800 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1f978-a7ee-4332-b3b8-177010ef21a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:43.621073 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1f978-a7ee-4332-b3b8-177010ef21a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:43.670476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1f978-a7ee-4332-b3b8-177010ef21a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:43.693017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-72a1f978-a7ee-4332-b3b8-177010ef21a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 15:59:43.693017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:43.693017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:43.693017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:43.693017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:43.693017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:43.693017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:43.693017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:43.693017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:43.693017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:43.693017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:43.693017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:43.693017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:43.693017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:43.693017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:43.693017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:43.694886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:43.694886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:43.694886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:43.694886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:43.694886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:43.694886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:43.694886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:43.694886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:43.694886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:43.694886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:43.694886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:43.694886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:43.694886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:43.694886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:43.694886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:43.694886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:43.696501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:43.696501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:43.696501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:43.696501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:43.696501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:43.696501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:43.696501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:43.696501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:43.696501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:43.696501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:43.696501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 15:59:43.696501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 15:59:43.700109 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 58/118] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 15:59:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:43.706726 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a2a1d-4cc1-49a1-b0e1-3c9a5bfd2dd3 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:43.706726 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a2a1d-4cc1-49a1-b0e1-3c9a5bfd2dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:43.706726 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a2a1d-4cc1-49a1-b0e1-3c9a5bfd2dd3 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:43.706726 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 59/119] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:43.715133 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a2a1d-4cc1-49a1-b0e1-3c9a5bfd2dd3 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:43.731807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:43.853894 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a2a1d-4cc1-49a1-b0e1-3c9a5bfd2dd3 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:43.856228 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7316280-182a-4436-be23-d7c8233dee83 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:43.911041 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e727e27-ada8-481e-9b9d-dd8984a9cfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:43.912207 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e727e27-ada8-481e-9b9d-dd8984a9cfa7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:43.912705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e727e27-ada8-481e-9b9d-dd8984a9cfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:43.912857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852daf7-1c1f-4832-944b-4ae8bbf61872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:43.913200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e727e27-ada8-481e-9b9d-dd8984a9cfa7 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:43.913999 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852daf7-1c1f-4832-944b-4ae8bbf61872 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:43.914286 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852daf7-1c1f-4832-944b-4ae8bbf61872 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:43.918699 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852daf7-1c1f-4832-944b-4ae8bbf61872 None admin] PATH_INFO: `/v3/users/feda116f177c415fa220633084d10d2f` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:43.918927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e727e27-ada8-481e-9b9d-dd8984a9cfa7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:43.918927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e727e27-ada8-481e-9b9d-dd8984a9cfa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:43.919184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852daf7-1c1f-4832-944b-4ae8bbf61872 None admin] RBAC: Authorizing `identity:get_user(user_id=feda116f177c415fa220633084d10d2f)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:43.919184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852daf7-1c1f-4832-944b-4ae8bbf61872 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:43.919438 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e727e27-ada8-481e-9b9d-dd8984a9cfa7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:43.919573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 61/120] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:43 2022] GET /identity/v3/users/feda116f177c415fa220633084d10d2f => generated 247 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:43.920157 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852daf7-1c1f-4832-944b-4ae8bbf61872 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:43.936746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 60/121] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 15:59:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 15:59:43.939507 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f00c5-a0b4-4eb1-a125-ccb9bc57059d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:43.939982 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f00c5-a0b4-4eb1-a125-ccb9bc57059d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:43.940423 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f00c5-a0b4-4eb1-a125-ccb9bc57059d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:43.974759 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d16f00c5-a0b4-4eb1-a125-ccb9bc57059d None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:43.977444 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d16f00c5-a0b4-4eb1-a125-ccb9bc57059d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:43.991687 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:44.095558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852daf7-1c1f-4832-944b-4ae8bbf61872 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:44.106907 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:44.117092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 61/122] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:44.120317 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f00c5-a0b4-4eb1-a125-ccb9bc57059d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:44.148218 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35a990-ab2a-453c-84a7-85fe86d5689f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:44.149459 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35a990-ab2a-453c-84a7-85fe86d5689f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:44.151580 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35a990-ab2a-453c-84a7-85fe86d5689f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:44.152090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35a990-ab2a-453c-84a7-85fe86d5689f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:44.175369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c35a990-ab2a-453c-84a7-85fe86d5689f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:44.182307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-5c35a990-ab2a-453c-84a7-85fe86d5689f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 15:59:44.182307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:44.182307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:44.182307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:44.182307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:44.182307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:44.182307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:44.182307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:44.182307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:44.182307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:44.182307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:44.182307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:44.182307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:44.182307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:44.182307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:44.182307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:44.183289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:44.183289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:44.183289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:44.183289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:44.183289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:44.183289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:44.183289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:44.183289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:44.183289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:44.183289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:44.183289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:44.183289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:44.183289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:44.183289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:44.183289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:44.183289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:44.184041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:44.184041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:44.184041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:44.184041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:44.184041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:44.184041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:44.184041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:44.184041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:44.184041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:44.184041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:44.184041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 15:59:44.184041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 15:59:44.191953 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 62/123] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 15:59:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:44.198767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35a990-ab2a-453c-84a7-85fe86d5689f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:44.451392 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f00c5-a0b4-4eb1-a125-ccb9bc57059d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:44.455070 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35a990-ab2a-453c-84a7-85fe86d5689f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:44.484806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:44.529024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24cd2c-86ef-4437-b779-6a098e99d6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:44.529691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24cd2c-86ef-4437-b779-6a098e99d6f5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:44.529972 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24cd2c-86ef-4437-b779-6a098e99d6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:44.530251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24cd2c-86ef-4437-b779-6a098e99d6f5 None admin] PATH_INFO: `/v3/users/5bbe86494fef439facc3d7e23e115ded` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:44.531487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24cd2c-86ef-4437-b779-6a098e99d6f5 None admin] RBAC: Authorizing `identity:get_user(user_id=5bbe86494fef439facc3d7e23e115ded)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:44.532138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d805215-d0dd-4581-bd27-4efee71ef122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:44.532956 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24cd2c-86ef-4437-b779-6a098e99d6f5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:44.534372 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d805215-d0dd-4581-bd27-4efee71ef122 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:44.535006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 63/124] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:44 2022] GET /identity/v3/users/5bbe86494fef439facc3d7e23e115ded => generated 275 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:44.535413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24cd2c-86ef-4437-b779-6a098e99d6f5 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:44.536209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d805215-d0dd-4581-bd27-4efee71ef122 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:44.536779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d805215-d0dd-4581-bd27-4efee71ef122 None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:44.539440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d805215-d0dd-4581-bd27-4efee71ef122 None admin] RBAC: Authorizing `identity:get_project(project_id=dda7941c8f444d41afc018b93fcd36fe)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:44.542533 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d805215-d0dd-4581-bd27-4efee71ef122 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:44.546103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 62/125] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 15:59:44 2022] GET /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe => generated 298 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:44.547809 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d805215-d0dd-4581-bd27-4efee71ef122 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:44.646768 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:44.662926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24cd2c-86ef-4437-b779-6a098e99d6f5 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:44.693511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d805215-d0dd-4581-bd27-4efee71ef122 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:44.705638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897921ca-0504-4406-9b59-a3167aebc9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:44.706601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897921ca-0504-4406-9b59-a3167aebc9bc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:44.707110 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897921ca-0504-4406-9b59-a3167aebc9bc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:44.707541 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897921ca-0504-4406-9b59-a3167aebc9bc None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:44.708399 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897921ca-0504-4406-9b59-a3167aebc9bc None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:44.708877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897921ca-0504-4406-9b59-a3167aebc9bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:44.710514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897921ca-0504-4406-9b59-a3167aebc9bc None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:44.720619 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 64/126] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 15:59:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:44.727373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897921ca-0504-4406-9b59-a3167aebc9bc None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:44.737916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e63d00-d890-4e6d-bd87-d1f44fe4cf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:44.739668 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e63d00-d890-4e6d-bd87-d1f44fe4cf4c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:44.740137 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e63d00-d890-4e6d-bd87-d1f44fe4cf4c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:44.740570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e63d00-d890-4e6d-bd87-d1f44fe4cf4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:44.761622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e63d00-d890-4e6d-bd87-d1f44fe4cf4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:44.769624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-16e63d00-d890-4e6d-bd87-d1f44fe4cf4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 15:59:44.769624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:44.769624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:44.769624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:44.769624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:44.769624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:44.769624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:44.769624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:44.769624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:44.769624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:44.769624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:44.769624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:44.769624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:44.769624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:44.769624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:44.769624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:44.770762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:44.770762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:44.770762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:44.770762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:44.770762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:44.770762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:44.770762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:44.770762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:44.770762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:44.770762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:44.770762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:44.770762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:44.770762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:44.770762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:44.770762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:44.770762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:44.771510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:44.771510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:44.771510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:44.771510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:44.771510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:44.771510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:44.771510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:44.771510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:44.771510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:44.771510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:44.771510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 15:59:44.771510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 15:59:44.772075 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 63/127] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 15:59:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:44.777366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e63d00-d890-4e6d-bd87-d1f44fe4cf4c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:44.797281 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:44.864515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897921ca-0504-4406-9b59-a3167aebc9bc None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:44.903978 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e63d00-d890-4e6d-bd87-d1f44fe4cf4c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:44.905355 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541b1a0-8732-4a3b-9cca-2386e1e1d0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:44.906265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541b1a0-8732-4a3b-9cca-2386e1e1d0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:44.906789 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541b1a0-8732-4a3b-9cca-2386e1e1d0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:44.907263 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541b1a0-8732-4a3b-9cca-2386e1e1d0c0 None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:44.908745 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541b1a0-8732-4a3b-9cca-2386e1e1d0c0 None admin] RBAC: Authorizing `identity:get_project(project_id=dda7941c8f444d41afc018b93fcd36fe)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:44.910465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541b1a0-8732-4a3b-9cca-2386e1e1d0c0 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:44.912442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 65/128] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 15:59:44 2022] GET /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:44.914130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541b1a0-8732-4a3b-9cca-2386e1e1d0c0 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:44.941459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:44.952023 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb12165-b327-4f89-98e9-8af5501ff1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:44.952784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb12165-b327-4f89-98e9-8af5501ff1b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:44.953051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb12165-b327-4f89-98e9-8af5501ff1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:44.953336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb12165-b327-4f89-98e9-8af5501ff1b0 None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe/users/feda116f177c415fa220633084d10d2f/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:44.955542 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb12165-b327-4f89-98e9-8af5501ff1b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=dda7941c8f444d41afc018b93fcd36fe, user_id=feda116f177c415fa220633084d10d2f, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:44.957165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb12165-b327-4f89-98e9-8af5501ff1b0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:44.977079 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 64/129] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 15:59:44 2022] PUT /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe/users/feda116f177c415fa220633084d10d2f/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 15:59:44.977742 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb12165-b327-4f89-98e9-8af5501ff1b0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:45.039723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541b1a0-8732-4a3b-9cca-2386e1e1d0c0 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:45.057134 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:45.086840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b2f91-c9e6-4ba4-8525-1c000840ac8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:45.087916 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b2f91-c9e6-4ba4-8525-1c000840ac8f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:45.088431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b2f91-c9e6-4ba4-8525-1c000840ac8f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:45.088921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b2f91-c9e6-4ba4-8525-1c000840ac8f None admin] PATH_INFO: `/v3/users/5bbe86494fef439facc3d7e23e115ded` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:45.090739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b2f91-c9e6-4ba4-8525-1c000840ac8f None admin] RBAC: Authorizing `identity:get_user(user_id=5bbe86494fef439facc3d7e23e115ded)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:45.092762 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b2f91-c9e6-4ba4-8525-1c000840ac8f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:45.095054 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 66/130] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:44 2022] GET /identity/v3/users/5bbe86494fef439facc3d7e23e115ded => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:45.101371 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b2f91-c9e6-4ba4-8525-1c000840ac8f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:45.139346 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb12165-b327-4f89-98e9-8af5501ff1b0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:45.193734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20cc95-6024-4140-abe9-3a7b82acff93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:45.194391 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20cc95-6024-4140-abe9-3a7b82acff93 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:45.194778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20cc95-6024-4140-abe9-3a7b82acff93 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:45.195067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20cc95-6024-4140-abe9-3a7b82acff93 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:45.196010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20cc95-6024-4140-abe9-3a7b82acff93 None admin] RBAC: Adding query filter params (name=member) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:45.196330 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20cc95-6024-4140-abe9-3a7b82acff93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:45.198707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20cc95-6024-4140-abe9-3a7b82acff93 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:45.213753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 65/131] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 15:59:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:45.214570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20cc95-6024-4140-abe9-3a7b82acff93 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:45.245670 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b2f91-c9e6-4ba4-8525-1c000840ac8f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:45.290513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffc5b7-e2b2-4966-8fab-9882a8800d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:45.291658 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffc5b7-e2b2-4966-8fab-9882a8800d59 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:45.292446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffc5b7-e2b2-4966-8fab-9882a8800d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:45.293555 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffc5b7-e2b2-4966-8fab-9882a8800d59 None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe/users/5bbe86494fef439facc3d7e23e115ded/roles/bad9ff35005240e99bedef3fddb1dc77` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:45.296836 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffc5b7-e2b2-4966-8fab-9882a8800d59 None admin] RBAC: Authorizing `identity:create_grant(project_id=dda7941c8f444d41afc018b93fcd36fe, user_id=5bbe86494fef439facc3d7e23e115ded, role_id=bad9ff35005240e99bedef3fddb1dc77)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:45.297853 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:45.300595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffc5b7-e2b2-4966-8fab-9882a8800d59 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:45.341967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 67/132] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 15:59:45 2022] PUT /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe/users/5bbe86494fef439facc3d7e23e115ded/roles/bad9ff35005240e99bedef3fddb1dc77 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 15:59:45.343251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffc5b7-e2b2-4966-8fab-9882a8800d59 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:45.381182 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20cc95-6024-4140-abe9-3a7b82acff93 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:45.433748 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0b27e-d718-4de7-93a3-d13741beb0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:45.435201 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0b27e-d718-4de7-93a3-d13741beb0fb None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:45.435737 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0b27e-d718-4de7-93a3-d13741beb0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:45.436185 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0b27e-d718-4de7-93a3-d13741beb0fb None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:45.437969 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0b27e-d718-4de7-93a3-d13741beb0fb None admin] RBAC: Authorizing `identity:get_project(project_id=dda7941c8f444d41afc018b93fcd36fe)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:45.440431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0b27e-d718-4de7-93a3-d13741beb0fb None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:45.442992 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 66/133] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 15:59:45 2022] GET /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:45.451307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0b27e-d718-4de7-93a3-d13741beb0fb None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:45.457404 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:45.591723 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0b27e-d718-4de7-93a3-d13741beb0fb None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:45.618845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:45.642391 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee297ff-9d7b-4d78-b699-a7e70ee6226e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:45.643994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee297ff-9d7b-4d78-b699-a7e70ee6226e None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:45.644521 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee297ff-9d7b-4d78-b699-a7e70ee6226e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:45.644955 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee297ff-9d7b-4d78-b699-a7e70ee6226e None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe/users/5bbe86494fef439facc3d7e23e115ded/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:45.647664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee297ff-9d7b-4d78-b699-a7e70ee6226e None admin] RBAC: Authorizing `identity:create_grant(project_id=dda7941c8f444d41afc018b93fcd36fe, user_id=5bbe86494fef439facc3d7e23e115ded, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:45.649799 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee297ff-9d7b-4d78-b699-a7e70ee6226e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:45.669190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 67/134] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 15:59:45 2022] PUT /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe/users/5bbe86494fef439facc3d7e23e115ded/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 15:59:45.679776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0ba86-227c-43f5-94fa-ac7a97ef8250 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:45.680279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0ba86-227c-43f5-94fa-ac7a97ef8250 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:45.680770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0ba86-227c-43f5-94fa-ac7a97ef8250 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:45.683606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 68/135] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 15:59:45.693187 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312439a-70d5-4772-a3af-7dd44c7533c5 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:45.693683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312439a-70d5-4772-a3af-7dd44c7533c5 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:45.694166 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312439a-70d5-4772-a3af-7dd44c7533c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:45.703369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffc5b7-e2b2-4966-8fab-9882a8800d59 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:45.732220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-5312439a-70d5-4772-a3af-7dd44c7533c5 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:45.733859 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5312439a-70d5-4772-a3af-7dd44c7533c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:45.857091 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75b7ff-2d73-4fb6-b762-ebaf232fe745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:45.861423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75b7ff-2d73-4fb6-b762-ebaf232fe745 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:45.862114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75b7ff-2d73-4fb6-b762-ebaf232fe745 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:45.862603 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75b7ff-2d73-4fb6-b762-ebaf232fe745 None admin] PATH_INFO: `/v3/users/5bbe86494fef439facc3d7e23e115ded` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:45.865127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75b7ff-2d73-4fb6-b762-ebaf232fe745 None admin] RBAC: Authorizing `identity:get_user(user_id=5bbe86494fef439facc3d7e23e115ded)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:45.870970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75b7ff-2d73-4fb6-b762-ebaf232fe745 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:45.875080 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 68/136] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:45 2022] GET /identity/v3/users/5bbe86494fef439facc3d7e23e115ded => generated 275 bytes in 533 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:45.881730 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75b7ff-2d73-4fb6-b762-ebaf232fe745 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:45.957039 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 69/137] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:46.025900 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df45ca-8a81-4a06-8600-dbab9d078d29 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:46.026403 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df45ca-8a81-4a06-8600-dbab9d078d29 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:46.026937 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df45ca-8a81-4a06-8600-dbab9d078d29 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:46.035452 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 70/138] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:46 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:46.041209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374edbc-122e-462e-bd87-0f5bfb4d814c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:46.041693 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374edbc-122e-462e-bd87-0f5bfb4d814c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:46.042158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374edbc-122e-462e-bd87-0f5bfb4d814c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:46.053769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75b7ff-2d73-4fb6-b762-ebaf232fe745 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:46.065536 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:46.079848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e374edbc-122e-462e-bd87-0f5bfb4d814c None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:46.082380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e374edbc-122e-462e-bd87-0f5bfb4d814c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:46.094629 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de613b-9a6c-46d6-852e-f1acb5651731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:46.095312 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de613b-9a6c-46d6-852e-f1acb5651731 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:46.095590 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de613b-9a6c-46d6-852e-f1acb5651731 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:46.095865 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de613b-9a6c-46d6-852e-f1acb5651731 None admin] PATH_INFO: `/v3/projects/36229525382149a3831f7f4dc0b0e575` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:46.097092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de613b-9a6c-46d6-852e-f1acb5651731 None admin] RBAC: Authorizing `identity:get_project(project_id=36229525382149a3831f7f4dc0b0e575)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:46.098639 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de613b-9a6c-46d6-852e-f1acb5651731 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:46.100440 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 69/139] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 15:59:45 2022] GET /identity/v3/projects/36229525382149a3831f7f4dc0b0e575 => generated 312 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 15:59:46.106802 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de613b-9a6c-46d6-852e-f1acb5651731 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:46.222025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 71/140] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 15:59:46.227352 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de613b-9a6c-46d6-852e-f1acb5651731 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:46.268243 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be6490-da97-4a80-9ec8-c0fc74df2e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:46.268925 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be6490-da97-4a80-9ec8-c0fc74df2e2a None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:46.269207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be6490-da97-4a80-9ec8-c0fc74df2e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:46.269498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be6490-da97-4a80-9ec8-c0fc74df2e2a None admin] PATH_INFO: `/v3/projects/36229525382149a3831f7f4dc0b0e575/users/5bbe86494fef439facc3d7e23e115ded/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:46.271614 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be6490-da97-4a80-9ec8-c0fc74df2e2a None admin] RBAC: Authorizing `identity:create_grant(project_id=36229525382149a3831f7f4dc0b0e575, user_id=5bbe86494fef439facc3d7e23e115ded, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:46.273235 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be6490-da97-4a80-9ec8-c0fc74df2e2a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:46.290327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 70/141] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 15:59:46 2022] PUT /identity/v3/projects/36229525382149a3831f7f4dc0b0e575/users/5bbe86494fef439facc3d7e23e115ded/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 15:59:46.734440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374edbc-122e-462e-bd87-0f5bfb4d814c None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:46.734978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be6490-da97-4a80-9ec8-c0fc74df2e2a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:46.792115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:46.811213 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:46.866388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374edbc-122e-462e-bd87-0f5bfb4d814c None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:46.909290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fa74a-6845-4fa7-81cd-d04892c8a932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:46.910161 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fa74a-6845-4fa7-81cd-d04892c8a932 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:46.910448 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fa74a-6845-4fa7-81cd-d04892c8a932 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:46.910753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fa74a-6845-4fa7-81cd-d04892c8a932 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:46.911903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322fa74a-6845-4fa7-81cd-d04892c8a932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:46.914169 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322fa74a-6845-4fa7-81cd-d04892c8a932 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:46.916869 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 72/142] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 15:59:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:46.922579 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fa74a-6845-4fa7-81cd-d04892c8a932 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:47.009627 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be6490-da97-4a80-9ec8-c0fc74df2e2a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:47.040992 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fa74a-6845-4fa7-81cd-d04892c8a932 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:47.063076 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:47.081392 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3a6f3-76da-47d9-9eee-8ff0f9a8a0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:47.082046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3a6f3-76da-47d9-9eee-8ff0f9a8a0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:47.082323 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3a6f3-76da-47d9-9eee-8ff0f9a8a0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:47.082586 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3a6f3-76da-47d9-9eee-8ff0f9a8a0d2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:47.098074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14027c-559f-4f32-bf05-862eb2b3ddd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:47.098724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14027c-559f-4f32-bf05-862eb2b3ddd0 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:47.100576 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3a6f3-76da-47d9-9eee-8ff0f9a8a0d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:47.103092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14027c-559f-4f32-bf05-862eb2b3ddd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:47.103382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14027c-559f-4f32-bf05-862eb2b3ddd0 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:47.104109 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14027c-559f-4f32-bf05-862eb2b3ddd0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:47.105614 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14027c-559f-4f32-bf05-862eb2b3ddd0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:47.114515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-56f3a6f3-76da-47d9-9eee-8ff0f9a8a0d2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 15:59:47.114515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:47.114515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:47.114515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:47.114515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:47.114515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:47.114515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:47.114515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:47.114515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:47.114515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:47.114515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:47.114515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:47.114515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:47.114515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:47.114515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:47.114515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:47.116212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:47.116212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:47.116212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:47.116212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:47.116212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:47.116212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:47.116212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:47.116212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:47.116212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:47.116212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:47.116212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:47.116212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:47.116212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:47.116212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:47.116212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:47.116212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:47.117889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:47.117889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:47.117889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:47.117889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:47.117889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:47.117889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:47.117889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:47.117889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:47.117889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:47.117889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:47.117889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 15:59:47.117889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 15:59:47.120222 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 71/143] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 15:59:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:47.129707 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3a6f3-76da-47d9-9eee-8ff0f9a8a0d2 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:47.147167 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 73/144] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 6 15:59:46 2022] POST /identity/v3/users => generated 283 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 15:59:47.295319 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:47.302155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3a6f3-76da-47d9-9eee-8ff0f9a8a0d2 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:47.343638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6b25e-2903-45aa-af3a-86e11ce26096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:47.344286 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6b25e-2903-45aa-af3a-86e11ce26096 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:47.344561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6b25e-2903-45aa-af3a-86e11ce26096 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:47.344830 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6b25e-2903-45aa-af3a-86e11ce26096 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:47.345521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6b25e-2903-45aa-af3a-86e11ce26096 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:47.345822 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6b25e-2903-45aa-af3a-86e11ce26096 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:47.347506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6b25e-2903-45aa-af3a-86e11ce26096 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:47.358097 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 72/145] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 15:59:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:47.367210 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14027c-559f-4f32-bf05-862eb2b3ddd0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:47.520807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14027c-559f-4f32-bf05-862eb2b3ddd0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:47.560826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69277e1-4579-4131-9552-db8155f2a5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:47.561917 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69277e1-4579-4131-9552-db8155f2a5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:47.562200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69277e1-4579-4131-9552-db8155f2a5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:47.562490 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69277e1-4579-4131-9552-db8155f2a5e9 None admin] PATH_INFO: `/v3/users/01f72ea34d5f4ee5b72408d14e925ffe` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:47.563755 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69277e1-4579-4131-9552-db8155f2a5e9 None admin] RBAC: Authorizing `identity:get_user(user_id=01f72ea34d5f4ee5b72408d14e925ffe)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:47.565252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69277e1-4579-4131-9552-db8155f2a5e9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:47.566934 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 74/146] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:47 2022] GET /identity/v3/users/01f72ea34d5f4ee5b72408d14e925ffe => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:47.578935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6b25e-2903-45aa-af3a-86e11ce26096 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:47.753860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6b25e-2903-45aa-af3a-86e11ce26096 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:47.790524 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fd970-c906-45da-adb3-004546c7127a None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:47.791058 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fd970-c906-45da-adb3-004546c7127a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:47.791058 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fd970-c906-45da-adb3-004546c7127a None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:47.792588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 75/147] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 15:59:47.798127 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d083a8a-0fb6-4bc0-9f49-3a0b4d9addc0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:47.798419 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d083a8a-0fb6-4bc0-9f49-3a0b4d9addc0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:47.798735 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d083a8a-0fb6-4bc0-9f49-3a0b4d9addc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:47.811008 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85cf3f-c592-48b1-b80e-5eefcbf14c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:47.811900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85cf3f-c592-48b1-b80e-5eefcbf14c03 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:47.812318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85cf3f-c592-48b1-b80e-5eefcbf14c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:47.812747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85cf3f-c592-48b1-b80e-5eefcbf14c03 None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:47.814452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85cf3f-c592-48b1-b80e-5eefcbf14c03 None admin] RBAC: Authorizing `identity:get_project(project_id=dda7941c8f444d41afc018b93fcd36fe)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:47.816287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85cf3f-c592-48b1-b80e-5eefcbf14c03 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:47.818519 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 73/148] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 15:59:47 2022] GET /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:47.820946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1d083a8a-0fb6-4bc0-9f49-3a0b4d9addc0 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:47.822154 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1d083a8a-0fb6-4bc0-9f49-3a0b4d9addc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:47.827727 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85cf3f-c592-48b1-b80e-5eefcbf14c03 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:47.850179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:47.878396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:47.908304 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 76/149] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:48.019163 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85cf3f-c592-48b1-b80e-5eefcbf14c03 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:48.070994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8d85f-da98-49a5-b72c-b0619cdc2258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:48.071587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8d85f-da98-49a5-b72c-b0619cdc2258 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:48.071864 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8d85f-da98-49a5-b72c-b0619cdc2258 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:48.072139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8d85f-da98-49a5-b72c-b0619cdc2258 None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe/users/01f72ea34d5f4ee5b72408d14e925ffe/roles/11b32861fcd8469382d1a7047a95ea70` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:48.078635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8d85f-da98-49a5-b72c-b0619cdc2258 None admin] RBAC: Authorizing `identity:create_grant(project_id=dda7941c8f444d41afc018b93fcd36fe, user_id=01f72ea34d5f4ee5b72408d14e925ffe, role_id=11b32861fcd8469382d1a7047a95ea70)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:48.080137 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8d85f-da98-49a5-b72c-b0619cdc2258 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:48.104454 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 74/150] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 15:59:47 2022] PUT /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe/users/01f72ea34d5f4ee5b72408d14e925ffe/roles/11b32861fcd8469382d1a7047a95ea70 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 15:59:48.270875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d083a8a-0fb6-4bc0-9f49-3a0b4d9addc0 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:48.408054 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d083a8a-0fb6-4bc0-9f49-3a0b4d9addc0 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:48.473351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668c069-2188-4d5b-ac2a-5fb81913083f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:48.474384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668c069-2188-4d5b-ac2a-5fb81913083f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:48.474903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668c069-2188-4d5b-ac2a-5fb81913083f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:48.475375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668c069-2188-4d5b-ac2a-5fb81913083f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:48.490894 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2668c069-2188-4d5b-ac2a-5fb81913083f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:48.496884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-2668c069-2188-4d5b-ac2a-5fb81913083f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 15:59:48.496884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:48.496884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:48.496884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:48.496884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:48.496884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:48.496884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:48.496884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:48.496884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:48.496884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:48.496884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:48.496884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:48.496884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:48.496884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:48.496884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:48.496884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:48.497785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:48.497785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:48.497785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:48.497785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:48.497785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:48.497785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:48.497785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:48.497785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:48.497785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:48.497785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:48.497785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:48.497785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:48.497785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:48.497785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:48.497785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:48.497785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:48.498537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:48.498537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:48.498537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:48.498537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:48.498537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:48.498537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:48.498537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:48.498537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:48.498537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:48.498537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:48.498537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 15:59:48.498537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 15:59:48.501303 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 77/151] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 15:59:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:48.508739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8d85f-da98-49a5-b72c-b0619cdc2258 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:48.652185 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:48.670179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8d85f-da98-49a5-b72c-b0619cdc2258 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:48.715242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15141686-4761-4aba-a2eb-4e99dd13c9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:48.715924 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15141686-4761-4aba-a2eb-4e99dd13c9e6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:48.716220 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15141686-4761-4aba-a2eb-4e99dd13c9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:48.716476 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15141686-4761-4aba-a2eb-4e99dd13c9e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:48.717159 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15141686-4761-4aba-a2eb-4e99dd13c9e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:48.717563 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15141686-4761-4aba-a2eb-4e99dd13c9e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:48.719032 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15141686-4761-4aba-a2eb-4e99dd13c9e6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:48.732161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 75/152] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 15:59:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:48.738250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668c069-2188-4d5b-ac2a-5fb81913083f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:48.778116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:48.883894 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668c069-2188-4d5b-ac2a-5fb81913083f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:48.923736 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b00716-61d2-4f54-a3bb-20bcefbc2398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:48.924290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b00716-61d2-4f54-a3bb-20bcefbc2398 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:48.924823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b00716-61d2-4f54-a3bb-20bcefbc2398 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:48.925101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b00716-61d2-4f54-a3bb-20bcefbc2398 None admin] PATH_INFO: `/v3/users/feda116f177c415fa220633084d10d2f` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:48.926265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b00716-61d2-4f54-a3bb-20bcefbc2398 None admin] RBAC: Authorizing `identity:get_user(user_id=feda116f177c415fa220633084d10d2f)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:48.927757 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b00716-61d2-4f54-a3bb-20bcefbc2398 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:48.930232 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 78/153] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:48 2022] GET /identity/v3/users/feda116f177c415fa220633084d10d2f => generated 247 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:48.935988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15141686-4761-4aba-a2eb-4e99dd13c9e6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:49.092588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15141686-4761-4aba-a2eb-4e99dd13c9e6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:49.140712 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b1252-16b3-4826-9214-3351d8e41653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:49.141334 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b1252-16b3-4826-9214-3351d8e41653 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:49.141616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b1252-16b3-4826-9214-3351d8e41653 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:49.141883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b1252-16b3-4826-9214-3351d8e41653 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:49.171422 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:49.228175 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b1252-16b3-4826-9214-3351d8e41653 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:49.229588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b1252-16b3-4826-9214-3351d8e41653 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:49.344884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 76/154] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 15:59:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 15:59:49.351530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b00716-61d2-4f54-a3bb-20bcefbc2398 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:49.459191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:49.474219 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b00716-61d2-4f54-a3bb-20bcefbc2398 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:49.517112 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9208460-d66a-448c-9851-ca230139fcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:49.517670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9208460-d66a-448c-9851-ca230139fcd9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:49.517944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9208460-d66a-448c-9851-ca230139fcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:49.518211 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9208460-d66a-448c-9851-ca230139fcd9 None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:49.523661 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9208460-d66a-448c-9851-ca230139fcd9 None admin] RBAC: Authorizing `identity:get_project(project_id=dda7941c8f444d41afc018b93fcd36fe)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:49.525210 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9208460-d66a-448c-9851-ca230139fcd9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:49.527975 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 79/155] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 15:59:49 2022] GET /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:49.533995 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b1252-16b3-4826-9214-3351d8e41653 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:49.619230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:49.687381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b1252-16b3-4826-9214-3351d8e41653 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:49.732342 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ef9a8-a556-4a5f-b249-6d539bb777a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:49.733057 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ef9a8-a556-4a5f-b249-6d539bb777a0 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:49.733220 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ef9a8-a556-4a5f-b249-6d539bb777a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:49.733496 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ef9a8-a556-4a5f-b249-6d539bb777a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:49.734238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ef9a8-a556-4a5f-b249-6d539bb777a0 None admin] RBAC: Adding query filter params (role.id=ba05677650e046a48d5f19440d48fbf2, scope.project.id=dda7941c8f444d41afc018b93fcd36fe, user.id=feda116f177c415fa220633084d10d2f) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:49.734545 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ef9a8-a556-4a5f-b249-6d539bb777a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:49.736024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ef9a8-a556-4a5f-b249-6d539bb777a0 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:49.743807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 77/156] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 15:59:49 2022] GET /identity/v3/role_assignments?user.id=feda116f177c415fa220633084d10d2f&scope.project.id=dda7941c8f444d41afc018b93fcd36fe&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:50.558523 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a852e6-e073-4106-a692-7c4831dcd30d None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:50.560211 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a852e6-e073-4106-a692-7c4831dcd30d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:50.569139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a852e6-e073-4106-a692-7c4831dcd30d None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:50.576649 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 80/157] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:50 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:50.582596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbeb03-cd74-4c9b-b7dc-150cfd1e49c2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:50.583399 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbeb03-cd74-4c9b-b7dc-150cfd1e49c2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:50.583399 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbeb03-cd74-4c9b-b7dc-150cfd1e49c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:50.616396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e1cbeb03-cd74-4c9b-b7dc-150cfd1e49c2 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:50.618020 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e1cbeb03-cd74-4c9b-b7dc-150cfd1e49c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:50.680512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4630ec-d9a3-4f1c-af4d-d3b88eb4aacc None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:50.682310 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4630ec-d9a3-4f1c-af4d-d3b88eb4aacc None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:50.683055 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4630ec-d9a3-4f1c-af4d-d3b88eb4aacc None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:50.691802 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 81/158] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:50 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:50.700567 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5749e-a429-4984-952a-851882335a82 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:50.701892 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5749e-a429-4984-952a-851882335a82 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:50.702606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5749e-a429-4984-952a-851882335a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:50.749212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 78/159] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:50.756439 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8ac5749e-a429-4984-952a-851882335a82 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:50.759018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8ac5749e-a429-4984-952a-851882335a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:50.945080 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 82/160] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:51.205071 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84203f8e-df34-4bf2-b4b9-5aa926daeb7f None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:51.205741 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84203f8e-df34-4bf2-b4b9-5aa926daeb7f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:51.206229 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84203f8e-df34-4bf2-b4b9-5aa926daeb7f None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:51.207901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 79/161] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 15:59:51.217328 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269762e0-32a0-49f2-b015-d390d87e0bc0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:51.218335 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269762e0-32a0-49f2-b015-d390d87e0bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:51.219099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269762e0-32a0-49f2-b015-d390d87e0bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:51.259544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-269762e0-32a0-49f2-b015-d390d87e0bc0 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:51.262023 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-269762e0-32a0-49f2-b015-d390d87e0bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:51.371446 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 83/162] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:51.490377 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84203f8e-df34-4bf2-b4b9-5aa926daeb7f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:51.635068 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:51.749539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84203f8e-df34-4bf2-b4b9-5aa926daeb7f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:51.828535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7cb699-e521-4673-9c3d-d030a355cfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:51.829625 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cb699-e521-4673-9c3d-d030a355cfb0 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:51.830125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cb699-e521-4673-9c3d-d030a355cfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:51.830614 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cb699-e521-4673-9c3d-d030a355cfb0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:51.855180 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7cb699-e521-4673-9c3d-d030a355cfb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:51.869123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-3c7cb699-e521-4673-9c3d-d030a355cfb0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 15:59:51.869123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:51.869123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:51.869123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:51.869123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:51.869123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:51.869123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:51.869123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:51.869123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:51.869123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:51.869123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:51.869123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:51.869123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:51.869123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:51.869123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:51.869123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:51.870051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:51.870051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:51.870051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:51.870051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:51.870051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:51.870051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:51.870051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:51.870051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:51.870051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:51.870051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:51.870051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:51.870051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:51.870051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:51.870051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:51.870051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:51.870051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:51.870940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:51.870940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:51.870940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:51.870940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:51.870940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:51.870940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:51.870940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:51.870940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:51.870940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:51.870940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:51.870940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 15:59:51.870940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 15:59:51.872081 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 80/163] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 15:59:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:51.880220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269762e0-32a0-49f2-b015-d390d87e0bc0 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:51.881313 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7cb699-e521-4673-9c3d-d030a355cfb0 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:51.939194 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:52.121275 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7cb699-e521-4673-9c3d-d030a355cfb0 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:52.123682 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:52.217739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee188433-1318-4f09-ab6f-d3d9508cf219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:52.218405 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee188433-1318-4f09-ab6f-d3d9508cf219 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:52.218703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee188433-1318-4f09-ab6f-d3d9508cf219 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:52.218983 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee188433-1318-4f09-ab6f-d3d9508cf219 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:52.247034 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee188433-1318-4f09-ab6f-d3d9508cf219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:52.252380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269762e0-32a0-49f2-b015-d390d87e0bc0 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:52.252769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-ee188433-1318-4f09-ab6f-d3d9508cf219 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 15:59:52.252769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:52.252769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:52.252769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:52.252769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:52.252769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:52.252769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:52.252769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:52.252769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:52.252769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:52.252769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:52.252769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:52.252769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:52.252769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:52.252769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:52.252769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:52.253581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:52.253581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:52.253581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:52.253581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:52.253581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:52.253581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:52.253581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:52.253581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:52.253581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:52.253581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:52.253581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:52.253581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:52.253581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:52.253581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:52.253581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:52.253581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:52.254358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:52.254358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:52.254358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:52.254358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:52.254358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:52.254358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:52.254358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:52.254358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:52.254358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:52.254358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:52.254358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 15:59:52.254358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 15:59:52.254358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 81/164] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 15:59:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:52.256076 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee188433-1318-4f09-ab6f-d3d9508cf219 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:52.329715 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9586a-7c83-4b44-91ec-7b0a63cac5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:52.330937 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9586a-7c83-4b44-91ec-7b0a63cac5fc None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:52.331489 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9586a-7c83-4b44-91ec-7b0a63cac5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:52.331973 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9586a-7c83-4b44-91ec-7b0a63cac5fc None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:52.332860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9586a-7c83-4b44-91ec-7b0a63cac5fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:52.333360 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9586a-7c83-4b44-91ec-7b0a63cac5fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:52.335106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9586a-7c83-4b44-91ec-7b0a63cac5fc None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:52.354924 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 84/165] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 15:59:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:52.356646 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9260009-ade8-4547-abac-ff32167c7000 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:52.357188 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9260009-ade8-4547-abac-ff32167c7000 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:52.357657 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9260009-ade8-4547-abac-ff32167c7000 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:52.359577 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 85/166] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:52.360544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9260009-ade8-4547-abac-ff32167c7000 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:52.474242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee188433-1318-4f09-ab6f-d3d9508cf219 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:52.509038 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:52.557123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3508e-5460-445c-a0e3-6e3ec6ac8a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:52.558226 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3508e-5460-445c-a0e3-6e3ec6ac8a96 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:52.558738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3508e-5460-445c-a0e3-6e3ec6ac8a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:52.559236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3508e-5460-445c-a0e3-6e3ec6ac8a96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:52.609505 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9260009-ade8-4547-abac-ff32167c7000 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:52.615699 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3508e-5460-445c-a0e3-6e3ec6ac8a96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:52.631075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-c7e3508e-5460-445c-a0e3-6e3ec6ac8a96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 15:59:52.631075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:52.631075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:52.631075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:52.631075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:52.631075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:52.631075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:52.631075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:52.631075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:52.631075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:52.631075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:52.631075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:52.631075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:52.631075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:52.631075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:52.631075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:52.632098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:52.632098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:52.632098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:52.632098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:52.632098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:52.632098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:52.632098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:52.632098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:52.632098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:52.632098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:52.632098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:52.632098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:52.632098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:52.632098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:52.632098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:52.632098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:52.632965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:52.632965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:52.632965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:52.632965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:52.632965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:52.632965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:52.632965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:52.632965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:52.632965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:52.632965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:52.632965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 15:59:52.632965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 15:59:52.635006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 82/167] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 15:59:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:52.636510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c9a66-eddc-47d5-b70e-ecbd3a7796a0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:52.637373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c9a66-eddc-47d5-b70e-ecbd3a7796a0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:52.637774 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c9a66-eddc-47d5-b70e-ecbd3a7796a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:52.664328 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-530c9a66-eddc-47d5-b70e-ecbd3a7796a0 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:52.665893 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-530c9a66-eddc-47d5-b70e-ecbd3a7796a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:52.708948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:52.726631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521dbf9-10f2-44a1-883f-671fe1376f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:52.727670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521dbf9-10f2-44a1-883f-671fe1376f5b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:52.728257 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521dbf9-10f2-44a1-883f-671fe1376f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:52.728745 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521dbf9-10f2-44a1-883f-671fe1376f5b None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:52.729974 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521dbf9-10f2-44a1-883f-671fe1376f5b None admin] RBAC: Adding query filter params (name=member) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:52.730533 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521dbf9-10f2-44a1-883f-671fe1376f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:52.733335 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521dbf9-10f2-44a1-883f-671fe1376f5b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:52.749269 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 86/168] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 15:59:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:52.750803 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521dbf9-10f2-44a1-883f-671fe1376f5b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:52.775813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 83/169] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:52.777534 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c9a66-eddc-47d5-b70e-ecbd3a7796a0 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:52.956595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521dbf9-10f2-44a1-883f-671fe1376f5b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:53.022687 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c9a66-eddc-47d5-b70e-ecbd3a7796a0 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:53.026751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf742efa-2aca-462f-8ad4-bc7fe0818f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:53.027706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf742efa-2aca-462f-8ad4-bc7fe0818f21 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:53.028187 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf742efa-2aca-462f-8ad4-bc7fe0818f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:53.028668 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf742efa-2aca-462f-8ad4-bc7fe0818f21 None admin] PATH_INFO: `/v3/users/5bbe86494fef439facc3d7e23e115ded` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:53.030087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf742efa-2aca-462f-8ad4-bc7fe0818f21 None admin] RBAC: Authorizing `identity:get_user(user_id=5bbe86494fef439facc3d7e23e115ded)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:53.031882 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf742efa-2aca-462f-8ad4-bc7fe0818f21 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:53.033865 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 87/170] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:52 2022] GET /identity/v3/users/5bbe86494fef439facc3d7e23e115ded => generated 275 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:53.035469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8152ec0-73e9-4995-8a4b-dbf5c3a97492 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:53.035982 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8152ec0-73e9-4995-8a4b-dbf5c3a97492 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:53.036450 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8152ec0-73e9-4995-8a4b-dbf5c3a97492 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:53.037971 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 88/171] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:53.039622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8152ec0-73e9-4995-8a4b-dbf5c3a97492 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:53.052131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:53.072846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f415b3-8396-45ff-94f8-e5ba5353969d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:53.073262 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f415b3-8396-45ff-94f8-e5ba5353969d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:53.073544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f415b3-8396-45ff-94f8-e5ba5353969d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:53.073808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f415b3-8396-45ff-94f8-e5ba5353969d None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:53.074490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f415b3-8396-45ff-94f8-e5ba5353969d None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:53.074847 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f415b3-8396-45ff-94f8-e5ba5353969d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:53.076237 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f415b3-8396-45ff-94f8-e5ba5353969d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:53.085914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 84/172] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 15:59:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:53.089544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2e44e-6ad0-4443-90ad-5f2d9603019c None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:53.089544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2e44e-6ad0-4443-90ad-5f2d9603019c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:53.089544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2e44e-6ad0-4443-90ad-5f2d9603019c None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:53.089544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 85/173] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:53.089925 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2e44e-6ad0-4443-90ad-5f2d9603019c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:53.113647 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:53.165931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8152ec0-73e9-4995-8a4b-dbf5c3a97492 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:53.204463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c920d31-0b08-4121-a2f1-e9c33faaad6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:53.205111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c920d31-0b08-4121-a2f1-e9c33faaad6f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:53.205388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c920d31-0b08-4121-a2f1-e9c33faaad6f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:53.205657 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c920d31-0b08-4121-a2f1-e9c33faaad6f None admin] PATH_INFO: `/v3/users/5bbe86494fef439facc3d7e23e115ded` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:53.206893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c920d31-0b08-4121-a2f1-e9c33faaad6f None admin] RBAC: Authorizing `identity:get_user(user_id=5bbe86494fef439facc3d7e23e115ded)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:53.208407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c920d31-0b08-4121-a2f1-e9c33faaad6f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:53.210214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 89/174] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:53 2022] GET /identity/v3/users/5bbe86494fef439facc3d7e23e115ded => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:53.212132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53743e99-de7e-4d36-a82d-252cf65bce50 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:53.212416 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53743e99-de7e-4d36-a82d-252cf65bce50 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:53.212686 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53743e99-de7e-4d36-a82d-252cf65bce50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:53.217747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2e44e-6ad0-4443-90ad-5f2d9603019c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:53.232428 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-53743e99-de7e-4d36-a82d-252cf65bce50 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:53.233638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-53743e99-de7e-4d36-a82d-252cf65bce50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:53.257998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d140ee-b5eb-4989-b67a-bde112c2e8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:53.258705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d140ee-b5eb-4989-b67a-bde112c2e8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:53.258984 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d140ee-b5eb-4989-b67a-bde112c2e8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:53.259261 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d140ee-b5eb-4989-b67a-bde112c2e8a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:53.315027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 90/175] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:53.320996 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24604242-3df4-4501-a96e-6a379b0c7992 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:53.321433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24604242-3df4-4501-a96e-6a379b0c7992 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:53.321875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24604242-3df4-4501-a96e-6a379b0c7992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:53.337221 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:53.345304 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-24604242-3df4-4501-a96e-6a379b0c7992 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:53.346510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-24604242-3df4-4501-a96e-6a379b0c7992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:53.362639 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d140ee-b5eb-4989-b67a-bde112c2e8a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:53.364611 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d140ee-b5eb-4989-b67a-bde112c2e8a3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:53.385951 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:53.442293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 91/176] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:53.443223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24604242-3df4-4501-a96e-6a379b0c7992 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:53.561621 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 86/177] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 15:59:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 15:59:53.563309 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d140ee-b5eb-4989-b67a-bde112c2e8a3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:53.588081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24604242-3df4-4501-a96e-6a379b0c7992 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:53.643148 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17a4d8-8f32-40e2-b2b2-99f5d183067a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:53.644566 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17a4d8-8f32-40e2-b2b2-99f5d183067a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:53.645199 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17a4d8-8f32-40e2-b2b2-99f5d183067a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:53.645841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17a4d8-8f32-40e2-b2b2-99f5d183067a None admin] PATH_INFO: `/v3/users/5bbe86494fef439facc3d7e23e115ded` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:53.648014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17a4d8-8f32-40e2-b2b2-99f5d183067a None admin] RBAC: Authorizing `identity:get_user(user_id=5bbe86494fef439facc3d7e23e115ded)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:53.651290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17a4d8-8f32-40e2-b2b2-99f5d183067a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:53.654404 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 92/178] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:53 2022] GET /identity/v3/users/5bbe86494fef439facc3d7e23e115ded => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:53.659462 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b397f1-2896-4c24-913b-4ecddd72eac3 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:53.660091 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b397f1-2896-4c24-913b-4ecddd72eac3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:53.660851 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b397f1-2896-4c24-913b-4ecddd72eac3 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:53.666009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 93/179] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:53.672537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b397f1-2896-4c24-913b-4ecddd72eac3 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:53.677932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:53.732217 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d140ee-b5eb-4989-b67a-bde112c2e8a3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:53.750800 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:53.777834 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b1171-6ad7-49f5-b008-42ff21123987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:53.778480 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b1171-6ad7-49f5-b008-42ff21123987 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:53.779087 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b1171-6ad7-49f5-b008-42ff21123987 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:53.779367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b1171-6ad7-49f5-b008-42ff21123987 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:53.793810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b1171-6ad7-49f5-b008-42ff21123987 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:53.798862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-de9b1171-6ad7-49f5-b008-42ff21123987 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 15:59:53.798862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:53.798862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:53.798862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:53.798862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:53.798862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:53.798862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:53.798862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:53.798862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:53.798862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:53.798862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:53.798862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:53.798862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:53.798862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:53.798862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:53.798862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:53.800268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:53.800268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:53.800268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:53.800268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:53.800268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:53.800268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:53.800268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:53.800268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:53.800268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:53.800268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:53.800268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:53.800268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:53.800268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:53.800268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:53.800268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:53.800268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:53.801755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:53.801755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:53.801755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:53.801755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:53.801755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:53.801755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:53.801755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:53.801755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:53.801755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:53.801755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:53.801755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 15:59:53.801755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 15:59:53.801755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 87/180] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 15:59:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:53.801755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b1171-6ad7-49f5-b008-42ff21123987 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:53.846284 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b397f1-2896-4c24-913b-4ecddd72eac3 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:53.887196 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e110e-0807-4314-8d2b-b9394d2bdde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:53.888144 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e110e-0807-4314-8d2b-b9394d2bdde7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:53.888533 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e110e-0807-4314-8d2b-b9394d2bdde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:53.889029 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e110e-0807-4314-8d2b-b9394d2bdde7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:53.914945 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b1171-6ad7-49f5-b008-42ff21123987 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:53.955574 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0978519-ec7b-4f2c-9727-f29814ae1790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:53.956208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0978519-ec7b-4f2c-9727-f29814ae1790 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:53.956486 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0978519-ec7b-4f2c-9727-f29814ae1790 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:53.956745 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0978519-ec7b-4f2c-9727-f29814ae1790 None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:53.957914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0978519-ec7b-4f2c-9727-f29814ae1790 None admin] RBAC: Authorizing `identity:get_project(project_id=dda7941c8f444d41afc018b93fcd36fe)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:53.959413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0978519-ec7b-4f2c-9727-f29814ae1790 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:53.961270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 88/181] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 15:59:53 2022] GET /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:53.962063 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0978519-ec7b-4f2c-9727-f29814ae1790 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:53.975461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e110e-0807-4314-8d2b-b9394d2bdde7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:53.977647 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e110e-0807-4314-8d2b-b9394d2bdde7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:53.980497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:54.014942 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:54.087060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0978519-ec7b-4f2c-9727-f29814ae1790 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:54.102642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 94/182] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 15:59:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 15:59:54.103626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e110e-0807-4314-8d2b-b9394d2bdde7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:54.124665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665c91e-2e99-4306-bfc2-517ed5972d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:54.125305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665c91e-2e99-4306-bfc2-517ed5972d73 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:54.125582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665c91e-2e99-4306-bfc2-517ed5972d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:54.125853 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665c91e-2e99-4306-bfc2-517ed5972d73 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:54.138752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665c91e-2e99-4306-bfc2-517ed5972d73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:54.146583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-8665c91e-2e99-4306-bfc2-517ed5972d73 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 15:59:54.146583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:54.146583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:54.146583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:54.146583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:54.146583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:54.146583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:54.146583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:54.146583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:54.146583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:54.146583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:54.146583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:54.146583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:54.146583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:54.146583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:54.146583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:54.148827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:54.148827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:54.148827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:54.148827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:54.148827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:54.148827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:54.148827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:54.148827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:54.148827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:54.148827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:54.148827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:54.148827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:54.148827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:54.148827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:54.148827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:54.148827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:54.149799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:54.149799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:54.149799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:54.149799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:54.149799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:54.149799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:54.149799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:54.149799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:54.149799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:54.149799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:54.149799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 15:59:54.149799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 15:59:54.149799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 89/183] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 15:59:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:54.149799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67a1f4-b4a6-4e22-9639-d712a64a5b9b None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:54.150492 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67a1f4-b4a6-4e22-9639-d712a64a5b9b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:54.150492 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67a1f4-b4a6-4e22-9639-d712a64a5b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:54.169048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-3d67a1f4-b4a6-4e22-9639-d712a64a5b9b None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:54.170274 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3d67a1f4-b4a6-4e22-9639-d712a64a5b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:54.220737 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e110e-0807-4314-8d2b-b9394d2bdde7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:54.223471 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:54.258229 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 90/184] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:54.258994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67a1f4-b4a6-4e22-9639-d712a64a5b9b None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:54.283955 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a38a42-6bce-48b8-b614-2ce31cd5e0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:54.284582 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a38a42-6bce-48b8-b614-2ce31cd5e0be None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:54.284876 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a38a42-6bce-48b8-b614-2ce31cd5e0be None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:54.285131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a38a42-6bce-48b8-b614-2ce31cd5e0be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:54.295717 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:54.368301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a38a42-6bce-48b8-b614-2ce31cd5e0be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:54.369947 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a38a42-6bce-48b8-b614-2ce31cd5e0be None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:54.384373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67a1f4-b4a6-4e22-9639-d712a64a5b9b None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:54.433300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced1417-3a20-495f-9ba5-e60f1178c265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:54.434138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced1417-3a20-495f-9ba5-e60f1178c265 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:54.434525 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced1417-3a20-495f-9ba5-e60f1178c265 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:54.434872 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced1417-3a20-495f-9ba5-e60f1178c265 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:54.436111 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced1417-3a20-495f-9ba5-e60f1178c265 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:54.437899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced1417-3a20-495f-9ba5-e60f1178c265 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:54.440190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 91/185] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 15:59:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:54.440992 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced1417-3a20-495f-9ba5-e60f1178c265 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:54.521335 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 95/186] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 15:59:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 15:59:54.522358 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a38a42-6bce-48b8-b614-2ce31cd5e0be None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:54.526103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:54.577923 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced1417-3a20-495f-9ba5-e60f1178c265 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:54.580114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:54.621778 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d3e5c-179c-4f84-9eb9-82205ee9fe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:54.622320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d3e5c-179c-4f84-9eb9-82205ee9fe6d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:54.622595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d3e5c-179c-4f84-9eb9-82205ee9fe6d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:54.622896 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d3e5c-179c-4f84-9eb9-82205ee9fe6d None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:54.623572 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d3e5c-179c-4f84-9eb9-82205ee9fe6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:54.623871 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d3e5c-179c-4f84-9eb9-82205ee9fe6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:54.625294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d3e5c-179c-4f84-9eb9-82205ee9fe6d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:54.635654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 92/187] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 15:59:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:54.636522 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d3e5c-179c-4f84-9eb9-82205ee9fe6d None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:54.679545 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a38a42-6bce-48b8-b614-2ce31cd5e0be None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:54.721276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b0999-d997-4a00-85cf-55ea8a5141e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:54.721887 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b0999-d997-4a00-85cf-55ea8a5141e9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:54.722158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b0999-d997-4a00-85cf-55ea8a5141e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:54.722421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b0999-d997-4a00-85cf-55ea8a5141e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:54.723189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b0999-d997-4a00-85cf-55ea8a5141e9 None admin] RBAC: Adding query filter params (role.id=bad9ff35005240e99bedef3fddb1dc77, scope.project.id=dda7941c8f444d41afc018b93fcd36fe, user.id=5bbe86494fef439facc3d7e23e115ded) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:54.723510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b0999-d997-4a00-85cf-55ea8a5141e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:54.724954 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b0999-d997-4a00-85cf-55ea8a5141e9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:54.732467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 96/188] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 15:59:54 2022] GET /identity/v3/role_assignments?user.id=5bbe86494fef439facc3d7e23e115ded&scope.project.id=dda7941c8f444d41afc018b93fcd36fe&role.id=bad9ff35005240e99bedef3fddb1dc77 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:54.734887 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b0999-d997-4a00-85cf-55ea8a5141e9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:54.793605 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d3e5c-179c-4f84-9eb9-82205ee9fe6d None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:54.832132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:54.866060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585c162-abaa-4be3-a8e8-b9e13743be00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:54.867707 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585c162-abaa-4be3-a8e8-b9e13743be00 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:54.868153 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585c162-abaa-4be3-a8e8-b9e13743be00 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:54.868588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585c162-abaa-4be3-a8e8-b9e13743be00 None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:54.870632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585c162-abaa-4be3-a8e8-b9e13743be00 None admin] RBAC: Authorizing `identity:get_project(project_id=dda7941c8f444d41afc018b93fcd36fe)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:54.873438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585c162-abaa-4be3-a8e8-b9e13743be00 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:54.876557 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 93/189] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 15:59:54 2022] GET /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:54.878028 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585c162-abaa-4be3-a8e8-b9e13743be00 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:54.903561 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b0999-d997-4a00-85cf-55ea8a5141e9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:54.934354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:54.964075 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e97c3-6270-4e6b-acec-582cb6e9a29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:54.965051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e97c3-6270-4e6b-acec-582cb6e9a29e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:54.965526 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e97c3-6270-4e6b-acec-582cb6e9a29e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:54.965990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e97c3-6270-4e6b-acec-582cb6e9a29e None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:54.967154 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e97c3-6270-4e6b-acec-582cb6e9a29e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:54.967668 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e97c3-6270-4e6b-acec-582cb6e9a29e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:54.969964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e97c3-6270-4e6b-acec-582cb6e9a29e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:54.986049 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 97/190] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 15:59:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:54.990823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e97c3-6270-4e6b-acec-582cb6e9a29e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:55.036520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585c162-abaa-4be3-a8e8-b9e13743be00 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:55.088209 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e235c-4ec9-4e9d-8108-adbfd967daa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:55.088837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e235c-4ec9-4e9d-8108-adbfd967daa1 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:55.089122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e235c-4ec9-4e9d-8108-adbfd967daa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:55.089395 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e235c-4ec9-4e9d-8108-adbfd967daa1 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:55.090084 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e235c-4ec9-4e9d-8108-adbfd967daa1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:55.091637 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e235c-4ec9-4e9d-8108-adbfd967daa1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:55.125633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 94/191] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 6 15:59:54 2022] POST /identity/v3/users => generated 297 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 15:59:55.126404 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e235c-4ec9-4e9d-8108-adbfd967daa1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:55.181074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e97c3-6270-4e6b-acec-582cb6e9a29e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:55.235490 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:55.247996 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd788e53-36c6-4741-aea2-3bb04c7a5ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:55.249215 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd788e53-36c6-4741-aea2-3bb04c7a5ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:55.249702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd788e53-36c6-4741-aea2-3bb04c7a5ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:55.250189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd788e53-36c6-4741-aea2-3bb04c7a5ba5 None admin] PATH_INFO: `/v3/projects/36229525382149a3831f7f4dc0b0e575` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:55.252410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd788e53-36c6-4741-aea2-3bb04c7a5ba5 None admin] RBAC: Authorizing `identity:get_project(project_id=36229525382149a3831f7f4dc0b0e575)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:55.255219 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd788e53-36c6-4741-aea2-3bb04c7a5ba5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:55.259170 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 98/192] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 15:59:54 2022] GET /identity/v3/projects/36229525382149a3831f7f4dc0b0e575 => generated 312 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:55.260496 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd788e53-36c6-4741-aea2-3bb04c7a5ba5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:55.291301 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:55.316350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e235c-4ec9-4e9d-8108-adbfd967daa1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:55.376902 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b11124-54be-40c8-a844-85527f0fa5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:55.378348 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b11124-54be-40c8-a844-85527f0fa5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:55.383652 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b11124-54be-40c8-a844-85527f0fa5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:55.383652 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b11124-54be-40c8-a844-85527f0fa5d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:55.403153 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b11124-54be-40c8-a844-85527f0fa5d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:55.414052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-06b11124-54be-40c8-a844-85527f0fa5d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 15:59:55.414052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:55.414052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:55.414052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:55.414052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:55.414052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:55.414052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:55.414052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:55.414052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:55.414052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:55.414052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:55.414052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:55.414052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:55.414052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:55.414052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:55.414052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:55.415866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:55.415866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:55.415866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:55.415866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:55.415866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:55.415866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:55.415866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:55.415866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:55.415866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:55.415866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:55.415866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:55.415866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:55.415866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:55.415866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:55.415866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:55.415866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:55.417433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:55.417433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:55.417433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:55.417433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:55.417433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:55.417433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:55.417433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:55.417433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:55.417433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:55.417433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:55.417433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 15:59:55.417433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 15:59:55.417433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 95/193] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 15:59:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:55.422266 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b11124-54be-40c8-a844-85527f0fa5d3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:55.438424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd788e53-36c6-4741-aea2-3bb04c7a5ba5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:55.510241 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01032a66-bef5-48f8-8f6c-ce8db9cd0d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:55.510241 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01032a66-bef5-48f8-8f6c-ce8db9cd0d06 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:55.510241 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01032a66-bef5-48f8-8f6c-ce8db9cd0d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:55.510241 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01032a66-bef5-48f8-8f6c-ce8db9cd0d06 None admin] PATH_INFO: `/v3/users/cdc776676bb5450d92bf017aec1a3442` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:55.541179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01032a66-bef5-48f8-8f6c-ce8db9cd0d06 None admin] RBAC: Authorizing `identity:get_user(user_id=cdc776676bb5450d92bf017aec1a3442)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:55.543840 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01032a66-bef5-48f8-8f6c-ce8db9cd0d06 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:55.546928 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 99/194] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:55 2022] GET /identity/v3/users/cdc776676bb5450d92bf017aec1a3442 => generated 283 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:55.548309 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01032a66-bef5-48f8-8f6c-ce8db9cd0d06 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:55.611256 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b11124-54be-40c8-a844-85527f0fa5d3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:55.673610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eba9a1-a2eb-4040-9e98-7c00c58bf419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:55.674308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eba9a1-a2eb-4040-9e98-7c00c58bf419 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:55.675086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eba9a1-a2eb-4040-9e98-7c00c58bf419 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:55.675539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eba9a1-a2eb-4040-9e98-7c00c58bf419 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:55.676925 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eba9a1-a2eb-4040-9e98-7c00c58bf419 None admin] RBAC: Adding query filter params (role.id=e89162686187404da84651e0a3632b7b, scope.project.id=dda7941c8f444d41afc018b93fcd36fe, user.id=5bbe86494fef439facc3d7e23e115ded) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:55.677445 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eba9a1-a2eb-4040-9e98-7c00c58bf419 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:55.680806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eba9a1-a2eb-4040-9e98-7c00c58bf419 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:55.691562 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 96/195] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 15:59:55 2022] GET /identity/v3/role_assignments?user.id=5bbe86494fef439facc3d7e23e115ded&scope.project.id=dda7941c8f444d41afc018b93fcd36fe&role.id=e89162686187404da84651e0a3632b7b => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:55.694886 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eba9a1-a2eb-4040-9e98-7c00c58bf419 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:55.699887 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:55.737646 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:55.848487 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01032a66-bef5-48f8-8f6c-ce8db9cd0d06 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:55.926374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26921f69-c56c-45f1-8775-b0a216d24b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:55.928019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26921f69-c56c-45f1-8775-b0a216d24b9f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:55.929365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26921f69-c56c-45f1-8775-b0a216d24b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:55.930078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26921f69-c56c-45f1-8775-b0a216d24b9f None admin] PATH_INFO: `/v3/users/01f72ea34d5f4ee5b72408d14e925ffe` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:55.932831 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26921f69-c56c-45f1-8775-b0a216d24b9f None admin] RBAC: Authorizing `identity:get_user(user_id=01f72ea34d5f4ee5b72408d14e925ffe)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:55.935914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26921f69-c56c-45f1-8775-b0a216d24b9f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:55.939430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 100/196] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:55 2022] GET /identity/v3/users/01f72ea34d5f4ee5b72408d14e925ffe => generated 289 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:55.941462 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26921f69-c56c-45f1-8775-b0a216d24b9f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:56.021887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eba9a1-a2eb-4040-9e98-7c00c58bf419 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:56.111040 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7a7e5-d4f3-4561-b32b-6e9d062e3f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:56.112236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7a7e5-d4f3-4561-b32b-6e9d062e3f30 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:56.112729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7a7e5-d4f3-4561-b32b-6e9d062e3f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:56.113228 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7a7e5-d4f3-4561-b32b-6e9d062e3f30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:56.114647 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7a7e5-d4f3-4561-b32b-6e9d062e3f30 None admin] RBAC: Adding query filter params (role.id=e89162686187404da84651e0a3632b7b, scope.project.id=36229525382149a3831f7f4dc0b0e575, user.id=5bbe86494fef439facc3d7e23e115ded) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:56.115269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7a7e5-d4f3-4561-b32b-6e9d062e3f30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:56.118719 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7a7e5-d4f3-4561-b32b-6e9d062e3f30 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:56.126066 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26921f69-c56c-45f1-8775-b0a216d24b9f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:56.135276 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 97/197] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 15:59:55 2022] GET /identity/v3/role_assignments?user.id=5bbe86494fef439facc3d7e23e115ded&scope.project.id=36229525382149a3831f7f4dc0b0e575&role.id=e89162686187404da84651e0a3632b7b => generated 608 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:56.136918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7a7e5-d4f3-4561-b32b-6e9d062e3f30 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:56.155890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:56.175266 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c7acd-03e2-4654-b039-492ffff0efef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:56.176270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c7acd-03e2-4654-b039-492ffff0efef None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:56.176804 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c7acd-03e2-4654-b039-492ffff0efef None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:56.177272 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c7acd-03e2-4654-b039-492ffff0efef None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:56.180358 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c7acd-03e2-4654-b039-492ffff0efef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:56.181301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c7acd-03e2-4654-b039-492ffff0efef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:56.186704 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c7acd-03e2-4654-b039-492ffff0efef None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:56.203734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 101/198] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 15:59:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:56.206036 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c7acd-03e2-4654-b039-492ffff0efef None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:56.429478 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:56.470200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c7acd-03e2-4654-b039-492ffff0efef None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:56.499226 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7a7e5-d4f3-4561-b32b-6e9d062e3f30 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:56.553036 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ecdfd-43f9-43e1-99da-559fe8373e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:56.554325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ecdfd-43f9-43e1-99da-559fe8373e72 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:56.554805 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ecdfd-43f9-43e1-99da-559fe8373e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:56.555289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ecdfd-43f9-43e1-99da-559fe8373e72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:56.560462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221b65e-bcbd-4148-b5cd-7a556b1f3ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:56.561959 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221b65e-bcbd-4148-b5cd-7a556b1f3ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:56.562773 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221b65e-bcbd-4148-b5cd-7a556b1f3ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:56.563562 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221b65e-bcbd-4148-b5cd-7a556b1f3ee0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:56.624574 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:56.645671 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ecdfd-43f9-43e1-99da-559fe8373e72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:56.647492 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ecdfd-43f9-43e1-99da-559fe8373e72 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:56.716976 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221b65e-bcbd-4148-b5cd-7a556b1f3ee0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:56.718611 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221b65e-bcbd-4148-b5cd-7a556b1f3ee0 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:56.875556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:56.898125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 102/199] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 15:59:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 15:59:56.899721 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ecdfd-43f9-43e1-99da-559fe8373e72 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:56.930915 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 98/200] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 15:59:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 795 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 15:59:56.932577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221b65e-bcbd-4148-b5cd-7a556b1f3ee0 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:57.023337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:57.043207 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ecdfd-43f9-43e1-99da-559fe8373e72 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:57.086527 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7635a-6eac-464f-97b1-60184030f22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:57.087666 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7635a-6eac-464f-97b1-60184030f22f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:57.088096 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7635a-6eac-464f-97b1-60184030f22f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:57.088530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7635a-6eac-464f-97b1-60184030f22f None admin] PATH_INFO: `/v3/users/cdc776676bb5450d92bf017aec1a3442` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:57.090287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7635a-6eac-464f-97b1-60184030f22f None admin] RBAC: Authorizing `identity:get_user(user_id=cdc776676bb5450d92bf017aec1a3442)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:57.092231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7635a-6eac-464f-97b1-60184030f22f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:57.094191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 103/201] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:56 2022] GET /identity/v3/users/cdc776676bb5450d92bf017aec1a3442 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:57.095164 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7635a-6eac-464f-97b1-60184030f22f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:57.131242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221b65e-bcbd-4148-b5cd-7a556b1f3ee0 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:57.191443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717566e-73b3-4a24-92f8-9a9e41bca693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:57.192162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717566e-73b3-4a24-92f8-9a9e41bca693 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:57.192512 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717566e-73b3-4a24-92f8-9a9e41bca693 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:57.192721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717566e-73b3-4a24-92f8-9a9e41bca693 None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:57.193950 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717566e-73b3-4a24-92f8-9a9e41bca693 None admin] RBAC: Authorizing `identity:get_project(project_id=dda7941c8f444d41afc018b93fcd36fe)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:57.195858 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717566e-73b3-4a24-92f8-9a9e41bca693 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:57.202276 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 99/202] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 15:59:56 2022] GET /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:57.202276 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717566e-73b3-4a24-92f8-9a9e41bca693 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:57.237616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7635a-6eac-464f-97b1-60184030f22f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:57.252673 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:57.282779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c7dde-3e03-47c4-ba6e-85ddcd787f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:57.283351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c7dde-3e03-47c4-ba6e-85ddcd787f7d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:57.283629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c7dde-3e03-47c4-ba6e-85ddcd787f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:57.283901 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c7dde-3e03-47c4-ba6e-85ddcd787f7d None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:57.285254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c7dde-3e03-47c4-ba6e-85ddcd787f7d None admin] RBAC: Authorizing `identity:get_project(project_id=c57cd5c32d1447d3968cda9fc409de3c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:57.287027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c7dde-3e03-47c4-ba6e-85ddcd787f7d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:57.294251 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 104/203] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 15:59:57 2022] GET /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:57.295206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c7dde-3e03-47c4-ba6e-85ddcd787f7d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:57.327300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:57.332560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717566e-73b3-4a24-92f8-9a9e41bca693 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:57.378123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795bbdc-426b-41f4-bc99-9ebed245abfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:57.379046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795bbdc-426b-41f4-bc99-9ebed245abfd None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:57.379479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795bbdc-426b-41f4-bc99-9ebed245abfd None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:57.379972 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795bbdc-426b-41f4-bc99-9ebed245abfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:57.425619 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c7dde-3e03-47c4-ba6e-85ddcd787f7d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:57.470021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab944b5b-08cf-46e5-a810-4c9ba9ac82b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:57.470653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab944b5b-08cf-46e5-a810-4c9ba9ac82b8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:57.470980 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab944b5b-08cf-46e5-a810-4c9ba9ac82b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:57.471256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab944b5b-08cf-46e5-a810-4c9ba9ac82b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:57.471705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795bbdc-426b-41f4-bc99-9ebed245abfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:57.472003 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab944b5b-08cf-46e5-a810-4c9ba9ac82b8 None admin] RBAC: Adding query filter params (role.id=11b32861fcd8469382d1a7047a95ea70, scope.project.id=dda7941c8f444d41afc018b93fcd36fe, user.id=01f72ea34d5f4ee5b72408d14e925ffe) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:57.472312 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab944b5b-08cf-46e5-a810-4c9ba9ac82b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:57.473716 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795bbdc-426b-41f4-bc99-9ebed245abfd None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:57.473847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab944b5b-08cf-46e5-a810-4c9ba9ac82b8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:57.487217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 105/204] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 15:59:57 2022] GET /identity/v3/role_assignments?user.id=01f72ea34d5f4ee5b72408d14e925ffe&scope.project.id=dda7941c8f444d41afc018b93fcd36fe&role.id=11b32861fcd8469382d1a7047a95ea70 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:57.488246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab944b5b-08cf-46e5-a810-4c9ba9ac82b8 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:57.532668 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:57.609281 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 100/205] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 15:59:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 15:59:57.615854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795bbdc-426b-41f4-bc99-9ebed245abfd None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:57.639724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:57.659130 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab944b5b-08cf-46e5-a810-4c9ba9ac82b8 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:57.708427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a720d5-1973-48fc-8e14-f2e42682334b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:57.708427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a720d5-1973-48fc-8e14-f2e42682334b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:57.708427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a720d5-1973-48fc-8e14-f2e42682334b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:57.708427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a720d5-1973-48fc-8e14-f2e42682334b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:57.708858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a720d5-1973-48fc-8e14-f2e42682334b None admin] RBAC: Adding query filter params (role.id=ba05677650e046a48d5f19440d48fbf2, scope.project.id=c57cd5c32d1447d3968cda9fc409de3c, user.id=cdc776676bb5450d92bf017aec1a3442) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:57.708858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a720d5-1973-48fc-8e14-f2e42682334b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:57.708858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a720d5-1973-48fc-8e14-f2e42682334b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:57.713714 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 106/206] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 15:59:57 2022] GET /identity/v3/role_assignments?user.id=cdc776676bb5450d92bf017aec1a3442&scope.project.id=c57cd5c32d1447d3968cda9fc409de3c&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:57.793180 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795bbdc-426b-41f4-bc99-9ebed245abfd None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:57.839760 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c8076-6ccb-47b0-b4fd-e9b4f516c951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:57.840420 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c8076-6ccb-47b0-b4fd-e9b4f516c951 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:57.840773 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c8076-6ccb-47b0-b4fd-e9b4f516c951 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:57.841049 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c8076-6ccb-47b0-b4fd-e9b4f516c951 None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:57.842344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c8076-6ccb-47b0-b4fd-e9b4f516c951 None admin] RBAC: Authorizing `identity:get_project(project_id=c57cd5c32d1447d3968cda9fc409de3c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:57.843867 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c8076-6ccb-47b0-b4fd-e9b4f516c951 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:57.848718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 101/207] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 15:59:57 2022] GET /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 15:59:57.854185 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a720d5-1973-48fc-8e14-f2e42682334b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:57.885604 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b87e6-df80-41bf-8d07-c6c34c986d97 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:57.885912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b87e6-df80-41bf-8d07-c6c34c986d97 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:57.886142 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b87e6-df80-41bf-8d07-c6c34c986d97 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:57.887560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 102/208] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 15:59:57.893052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99525a76-3341-4f47-b189-6083ae22c680 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:57.893336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99525a76-3341-4f47-b189-6083ae22c680 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:57.893606 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99525a76-3341-4f47-b189-6083ae22c680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:57.915266 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-99525a76-3341-4f47-b189-6083ae22c680 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:57.916517 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-99525a76-3341-4f47-b189-6083ae22c680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:57.931060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:57.995145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a720d5-1973-48fc-8e14-f2e42682334b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:58.012670 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 103/209] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 15:59:58.014822 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc742d-38db-4d5f-98ef-6e1cf8b429f6 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:58.015337 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc742d-38db-4d5f-98ef-6e1cf8b429f6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:58.015808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc742d-38db-4d5f-98ef-6e1cf8b429f6 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:58.017804 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 104/210] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 15:59:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 15:59:58.024033 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b425490-26a6-4666-9075-ae20c436ff88 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:58.024363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b425490-26a6-4666-9075-ae20c436ff88 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:58.024586 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b425490-26a6-4666-9075-ae20c436ff88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:58.039062 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857cb03-3fe7-4ed7-9d8d-ab9c014d54b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:58.039716 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857cb03-3fe7-4ed7-9d8d-ab9c014d54b6 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:58.039999 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857cb03-3fe7-4ed7-9d8d-ab9c014d54b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:58.040274 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857cb03-3fe7-4ed7-9d8d-ab9c014d54b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:58.041017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857cb03-3fe7-4ed7-9d8d-ab9c014d54b6 None admin] RBAC: Adding query filter params (role.id=bad9ff35005240e99bedef3fddb1dc77, scope.project.id=c57cd5c32d1447d3968cda9fc409de3c, user.id=cdc776676bb5450d92bf017aec1a3442) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:58.041328 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857cb03-3fe7-4ed7-9d8d-ab9c014d54b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:58.042849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857cb03-3fe7-4ed7-9d8d-ab9c014d54b6 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:58.048836 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-5b425490-26a6-4666-9075-ae20c436ff88 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 15:59:58.050882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5b425490-26a6-4666-9075-ae20c436ff88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 15:59:58.055010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 107/211] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 15:59:57 2022] GET /identity/v3/role_assignments?user.id=cdc776676bb5450d92bf017aec1a3442&scope.project.id=c57cd5c32d1447d3968cda9fc409de3c&role.id=bad9ff35005240e99bedef3fddb1dc77 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:58.142846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 105/212] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 15:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 15:59:58.297827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857cb03-3fe7-4ed7-9d8d-ab9c014d54b6 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:58.327583 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:58.448743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b425490-26a6-4666-9075-ae20c436ff88 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:58.483876 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857cb03-3fe7-4ed7-9d8d-ab9c014d54b6 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:58.534035 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bd570-64db-4668-82f2-91f93070d04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:58.534704 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bd570-64db-4668-82f2-91f93070d04a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:58.534992 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bd570-64db-4668-82f2-91f93070d04a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:58.535274 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bd570-64db-4668-82f2-91f93070d04a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:58.545744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:58.549982 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12bd570-64db-4668-82f2-91f93070d04a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 15:59:58.555679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-b12bd570-64db-4668-82f2-91f93070d04a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 15:59:58.555679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 15:59:58.555679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 15:59:58.555679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 15:59:58.555679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 15:59:58.555679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 15:59:58.555679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 15:59:58.555679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 15:59:58.555679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 15:59:58.555679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 15:59:58.555679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 15:59:58.555679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 15:59:58.555679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 15:59:58.555679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 15:59:58.555679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 15:59:58.555679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 15:59:58.557345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 15:59:58.557345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 15:59:58.557345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 15:59:58.557345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 15:59:58.557345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 15:59:58.557345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 15:59:58.557345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 15:59:58.557345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 15:59:58.557345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 15:59:58.557345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 15:59:58.557345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 15:59:58.557345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 15:59:58.557345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 15:59:58.557345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 15:59:58.557345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 15:59:58.557345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 15:59:58.560197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 15:59:58.560197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 15:59:58.560197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 15:59:58.560197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 15:59:58.560197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 15:59:58.560197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 15:59:58.560197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 15:59:58.560197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 15:59:58.560197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 15:59:58.560197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 15:59:58.560197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 15:59:58.560197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 15:59:58.560197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 108/213] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 15:59:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 15:59:58.571876 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bd570-64db-4668-82f2-91f93070d04a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:58.590123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b425490-26a6-4666-9075-ae20c436ff88 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:58.629888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554eecf-47bd-4267-b5d2-b32f18c24425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:58.630593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554eecf-47bd-4267-b5d2-b32f18c24425 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:58.630908 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554eecf-47bd-4267-b5d2-b32f18c24425 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:58.631180 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554eecf-47bd-4267-b5d2-b32f18c24425 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:58.632977 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554eecf-47bd-4267-b5d2-b32f18c24425 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:58.634510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554eecf-47bd-4267-b5d2-b32f18c24425 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:58.636538 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 106/214] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 15:59:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 15:59:58.641883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554eecf-47bd-4267-b5d2-b32f18c24425 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:58.676258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:58.715788 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bd570-64db-4668-82f2-91f93070d04a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:58.767437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c0f63-dea2-4148-8ad0-0a4afb4adcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:58.768685 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c0f63-dea2-4148-8ad0-0a4afb4adcc7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:58.769449 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c0f63-dea2-4148-8ad0-0a4afb4adcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:58.770422 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c0f63-dea2-4148-8ad0-0a4afb4adcc7 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:58.771664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c0f63-dea2-4148-8ad0-0a4afb4adcc7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 15:59:58.772186 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c0f63-dea2-4148-8ad0-0a4afb4adcc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:58.774257 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554eecf-47bd-4267-b5d2-b32f18c24425 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:58.774633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c0f63-dea2-4148-8ad0-0a4afb4adcc7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:58.787430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 109/215] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 15:59:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 15:59:58.799872 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c0f63-dea2-4148-8ad0-0a4afb4adcc7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:58.823012 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5fdce-b6c1-4671-b974-ced464b40208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:58.823654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5fdce-b6c1-4671-b974-ced464b40208 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:58.823978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5fdce-b6c1-4671-b974-ced464b40208 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:58.824293 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5fdce-b6c1-4671-b974-ced464b40208 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:58.825014 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda5fdce-b6c1-4671-b974-ced464b40208 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:58.826455 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda5fdce-b6c1-4671-b974-ced464b40208 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:58.838495 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:58.862778 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 107/216] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 6 15:59:58 2022] POST /identity/v3/users => generated 297 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 15:59:58.952688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c0f63-dea2-4148-8ad0-0a4afb4adcc7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:58.999369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc3019-60d7-4aac-a8b1-dcf884a45f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:59.000414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc3019-60d7-4aac-a8b1-dcf884a45f66 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:59.001000 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc3019-60d7-4aac-a8b1-dcf884a45f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:59.001556 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc3019-60d7-4aac-a8b1-dcf884a45f66 None admin] PATH_INFO: `/v3/users/affc11afd71f4f4687e2259f7362c0fe` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:59.005544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:59.020579 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc3019-60d7-4aac-a8b1-dcf884a45f66 None admin] RBAC: Authorizing `identity:get_user(user_id=affc11afd71f4f4687e2259f7362c0fe)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:59.022577 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc3019-60d7-4aac-a8b1-dcf884a45f66 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:59.025102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 110/217] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 15:59:58 2022] GET /identity/v3/users/affc11afd71f4f4687e2259f7362c0fe => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:59.034290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5fdce-b6c1-4671-b974-ced464b40208 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:59.243295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5fdce-b6c1-4671-b974-ced464b40208 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:59.298922 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb5bef-7656-4b18-b655-997c227f127b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:59.300166 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb5bef-7656-4b18-b655-997c227f127b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:59.300659 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb5bef-7656-4b18-b655-997c227f127b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:59.301155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb5bef-7656-4b18-b655-997c227f127b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:59.420190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 15:59:59.443265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb5bef-7656-4b18-b655-997c227f127b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:59.446717 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb5bef-7656-4b18-b655-997c227f127b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:59.632101 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 108/218] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 15:59:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 15:59:59.638488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc3019-60d7-4aac-a8b1-dcf884a45f66 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 15:59:59.849632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc3019-60d7-4aac-a8b1-dcf884a45f66 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 15:59:59.928429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c8338-17a6-4227-bb23-142d71e6d612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 15:59:59.930747 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c8338-17a6-4227-bb23-142d71e6d612 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 15:59:59.931537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c8338-17a6-4227-bb23-142d71e6d612 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 15:59:59.932364 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c8338-17a6-4227-bb23-142d71e6d612 None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 15:59:59.934828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c8338-17a6-4227-bb23-142d71e6d612 None admin] RBAC: Authorizing `identity:get_project(project_id=c57cd5c32d1447d3968cda9fc409de3c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 15:59:59.937967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c8338-17a6-4227-bb23-142d71e6d612 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 15:59:59.942007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 111/219] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 15:59:59 2022] GET /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c => generated 302 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 15:59:59.956607 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb5bef-7656-4b18-b655-997c227f127b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:00.230678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:00.242717 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665700f-813b-49e3-8b4e-36f69868d588 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:00.243150 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665700f-813b-49e3-8b4e-36f69868d588 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:00.243299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665700f-813b-49e3-8b4e-36f69868d588 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:00.245019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 112/220] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:00:00.253524 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ab881-0104-4384-8fc2-73a3e39e2a02 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:00.253764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb5bef-7656-4b18-b655-997c227f127b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:00.254402 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ab881-0104-4384-8fc2-73a3e39e2a02 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:00.254402 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ab881-0104-4384-8fc2-73a3e39e2a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:00.294600 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-296ab881-0104-4384-8fc2-73a3e39e2a02 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:00.299426 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-296ab881-0104-4384-8fc2-73a3e39e2a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:00.326734 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6798a0-5c0f-4c9e-b0c1-ee10c0cba94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:00.327356 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6798a0-5c0f-4c9e-b0c1-ee10c0cba94b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:00.327635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6798a0-5c0f-4c9e-b0c1-ee10c0cba94b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:00.327902 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6798a0-5c0f-4c9e-b0c1-ee10c0cba94b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:00.328645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6798a0-5c0f-4c9e-b0c1-ee10c0cba94b None admin] RBAC: Adding query filter params (role.id=e89162686187404da84651e0a3632b7b, scope.project.id=c57cd5c32d1447d3968cda9fc409de3c, user.id=affc11afd71f4f4687e2259f7362c0fe) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:00.328955 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6798a0-5c0f-4c9e-b0c1-ee10c0cba94b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:00.330389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6798a0-5c0f-4c9e-b0c1-ee10c0cba94b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:00.349951 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 109/221] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 15:59:59 2022] GET /identity/v3/role_assignments?user.id=affc11afd71f4f4687e2259f7362c0fe&scope.project.id=c57cd5c32d1447d3968cda9fc409de3c&role.id=e89162686187404da84651e0a3632b7b => generated 259 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:00.412365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:00.430436 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 113/222] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:00:01.012321 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6798a0-5c0f-4c9e-b0c1-ee10c0cba94b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:01.064233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090d43c-bd42-4ef5-b2cb-19882494c47b None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:01.066206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090d43c-bd42-4ef5-b2cb-19882494c47b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:01.066767 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090d43c-bd42-4ef5-b2cb-19882494c47b None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:01.068709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 114/223] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:01.082814 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11960806-ffab-4107-af9b-5daeec3af254 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:01.083709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11960806-ffab-4107-af9b-5daeec3af254 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:01.084568 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11960806-ffab-4107-af9b-5daeec3af254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:01.127300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-11960806-ffab-4107-af9b-5daeec3af254 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:01.130746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-11960806-ffab-4107-af9b-5daeec3af254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:01.254372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6798a0-5c0f-4c9e-b0c1-ee10c0cba94b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:01.273054 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 115/224] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:01.353397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45731280-eb55-4cdb-a961-aedc2ad0e31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:01.354469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45731280-eb55-4cdb-a961-aedc2ad0e31f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:01.354997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45731280-eb55-4cdb-a961-aedc2ad0e31f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:01.355479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45731280-eb55-4cdb-a961-aedc2ad0e31f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:01.379657 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45731280-eb55-4cdb-a961-aedc2ad0e31f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:01.392016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-45731280-eb55-4cdb-a961-aedc2ad0e31f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:00:01.392016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:01.392016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:01.392016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:01.392016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:01.392016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:01.392016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:01.392016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:01.392016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:01.392016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:01.392016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:01.392016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:01.392016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:01.392016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:01.392016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:01.392016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:01.393726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:01.393726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:01.393726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:01.393726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:01.393726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:01.393726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:01.393726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:01.393726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:01.393726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:01.393726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:01.393726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:01.393726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:01.393726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:01.393726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:01.393726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:01.393726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:01.396194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:01.396194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:01.396194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:01.396194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:01.396194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:01.396194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:01.396194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:01.396194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:01.396194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:01.396194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:01.396194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:00:01.396194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:01.396194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 110/225] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:00:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:00:01.406995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11960806-ffab-4107-af9b-5daeec3af254 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:01.551613 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11960806-ffab-4107-af9b-5daeec3af254 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:01.579653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:01.601775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19306a92-32db-4308-8395-d97dc9257bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:01.601775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19306a92-32db-4308-8395-d97dc9257bda None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:01.601775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19306a92-32db-4308-8395-d97dc9257bda None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:01.601775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19306a92-32db-4308-8395-d97dc9257bda None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:01.602391 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19306a92-32db-4308-8395-d97dc9257bda None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:01.602391 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19306a92-32db-4308-8395-d97dc9257bda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:01.602922 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19306a92-32db-4308-8395-d97dc9257bda None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:01.619188 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 116/226] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:01.628508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45731280-eb55-4cdb-a961-aedc2ad0e31f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:01.713672 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:01.727000 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19306a92-32db-4308-8395-d97dc9257bda None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:01.850442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19306a92-32db-4308-8395-d97dc9257bda None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:01.901366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7105d-adf4-499e-afea-6afc64576ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:01.902023 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7105d-adf4-499e-afea-6afc64576ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:01.902301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7105d-adf4-499e-afea-6afc64576ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:01.902583 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7105d-adf4-499e-afea-6afc64576ab8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:01.921681 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7105d-adf4-499e-afea-6afc64576ab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:01.928139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-4ef7105d-adf4-499e-afea-6afc64576ab8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:00:01.928139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:01.928139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:01.928139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:01.928139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:01.928139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:01.928139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:01.928139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:01.928139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:01.928139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:01.928139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:01.928139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:01.928139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:01.928139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:01.928139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:01.928139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:01.929279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:01.929279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:01.929279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:01.929279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:01.929279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:01.929279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:01.929279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:01.929279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:01.929279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:01.929279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:01.929279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:01.929279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:01.929279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:01.929279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:01.929279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:01.929279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:01.930548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:01.930548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:01.930548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:01.930548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:01.930548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:01.930548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:01.930548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:01.930548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:01.930548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:01.930548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:01.930548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:00:01.930548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:01.930548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 117/227] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 16:00:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:00:01.940683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7105d-adf4-499e-afea-6afc64576ab8 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:01.955804 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45731280-eb55-4cdb-a961-aedc2ad0e31f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:02.036815 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa009944-09ce-4c0f-ab23-506d99749021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:02.037855 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa009944-09ce-4c0f-ab23-506d99749021 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:02.043083 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:02.043248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa009944-09ce-4c0f-ab23-506d99749021 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:02.043578 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa009944-09ce-4c0f-ab23-506d99749021 None admin] PATH_INFO: `/v3/users/cdc776676bb5450d92bf017aec1a3442` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:02.045825 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa009944-09ce-4c0f-ab23-506d99749021 None admin] RBAC: Authorizing `identity:get_user(user_id=cdc776676bb5450d92bf017aec1a3442)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:02.048902 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa009944-09ce-4c0f-ab23-506d99749021 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:02.052538 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 111/228] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:00:01 2022] GET /identity/v3/users/cdc776676bb5450d92bf017aec1a3442 => generated 283 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:02.072957 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa009944-09ce-4c0f-ab23-506d99749021 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:02.122187 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7105d-adf4-499e-afea-6afc64576ab8 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:02.171093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c566c79-03c1-469d-9455-8cff7a9aa470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:02.172106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c566c79-03c1-469d-9455-8cff7a9aa470 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:02.174176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c566c79-03c1-469d-9455-8cff7a9aa470 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:02.175060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c566c79-03c1-469d-9455-8cff7a9aa470 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:02.176400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c566c79-03c1-469d-9455-8cff7a9aa470 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:02.178227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c566c79-03c1-469d-9455-8cff7a9aa470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:02.183296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c566c79-03c1-469d-9455-8cff7a9aa470 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:02.210774 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 118/229] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:00:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:02.228591 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c566c79-03c1-469d-9455-8cff7a9aa470 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:02.301620 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:02.320632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa009944-09ce-4c0f-ab23-506d99749021 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:02.358546 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c566c79-03c1-469d-9455-8cff7a9aa470 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:02.360962 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60ce1f-bcfa-4c7d-afe3-dba5cf57e668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:02.362550 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60ce1f-bcfa-4c7d-afe3-dba5cf57e668 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:02.362868 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60ce1f-bcfa-4c7d-afe3-dba5cf57e668 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:02.363144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60ce1f-bcfa-4c7d-afe3-dba5cf57e668 None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:02.364354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60ce1f-bcfa-4c7d-afe3-dba5cf57e668 None admin] RBAC: Authorizing `identity:get_project(project_id=c57cd5c32d1447d3968cda9fc409de3c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:02.365828 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60ce1f-bcfa-4c7d-afe3-dba5cf57e668 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:02.368340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:02.369411 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 112/230] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:02 2022] GET /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:02.370576 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263c114-f9b8-494e-95f1-08e0ee2ec664 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:02.370914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263c114-f9b8-494e-95f1-08e0ee2ec664 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:02.371165 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263c114-f9b8-494e-95f1-08e0ee2ec664 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:02.377098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 113/231] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:02.377098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cb61c-9d2f-4919-81f8-847e1c342609 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:02.377098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cb61c-9d2f-4919-81f8-847e1c342609 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:02.377098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cb61c-9d2f-4919-81f8-847e1c342609 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:02.377098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 114/232] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:02.381257 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cb61c-9d2f-4919-81f8-847e1c342609 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:02.415429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc8503-077a-4a68-9bec-3673006bdc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:02.416854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc8503-077a-4a68-9bec-3673006bdc1f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:02.417543 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc8503-077a-4a68-9bec-3673006bdc1f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:02.418012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc8503-077a-4a68-9bec-3673006bdc1f None admin] PATH_INFO: `/v3/users/cdc776676bb5450d92bf017aec1a3442` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:02.420240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc8503-077a-4a68-9bec-3673006bdc1f None admin] RBAC: Authorizing `identity:get_user(user_id=cdc776676bb5450d92bf017aec1a3442)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:02.422969 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc8503-077a-4a68-9bec-3673006bdc1f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:02.431300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 119/233] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:00:02 2022] GET /identity/v3/users/cdc776676bb5450d92bf017aec1a3442 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:02.431300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40fc1ac-c42b-4293-8b7f-8823a1a9377c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:02.431300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40fc1ac-c42b-4293-8b7f-8823a1a9377c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:02.431300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40fc1ac-c42b-4293-8b7f-8823a1a9377c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:02.455528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-c40fc1ac-c42b-4293-8b7f-8823a1a9377c None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:02.456744 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c40fc1ac-c42b-4293-8b7f-8823a1a9377c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:02.515921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cb61c-9d2f-4919-81f8-847e1c342609 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:02.559397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bef57f-aa6b-40b4-b286-ddfe8a3c3234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:02.560182 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bef57f-aa6b-40b4-b286-ddfe8a3c3234 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:02.560474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bef57f-aa6b-40b4-b286-ddfe8a3c3234 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:02.561347 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bef57f-aa6b-40b4-b286-ddfe8a3c3234 None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c/users/cdc776676bb5450d92bf017aec1a3442/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:02.563553 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bef57f-aa6b-40b4-b286-ddfe8a3c3234 None admin] RBAC: Authorizing `identity:create_grant(project_id=c57cd5c32d1447d3968cda9fc409de3c, user_id=cdc776676bb5450d92bf017aec1a3442, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:02.565016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bef57f-aa6b-40b4-b286-ddfe8a3c3234 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:02.581246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 115/234] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:00:02 2022] PUT /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c/users/cdc776676bb5450d92bf017aec1a3442/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:00:02.582508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ead82-4280-469d-aaae-71531b2c7512 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:02.582836 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ead82-4280-469d-aaae-71531b2c7512 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:02.583092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ead82-4280-469d-aaae-71531b2c7512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:02.583680 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 120/235] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:02.587068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40fc1ac-c42b-4293-8b7f-8823a1a9377c None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:02.641920 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-194ead82-4280-469d-aaae-71531b2c7512 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:02.643764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-194ead82-4280-469d-aaae-71531b2c7512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:02.672511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:02.844074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40fc1ac-c42b-4293-8b7f-8823a1a9377c None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:02.844270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 116/236] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:02.851908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:02.878452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ead82-4280-469d-aaae-71531b2c7512 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:02.954761 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c1f1d-747b-41b2-9635-6dd397458920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:02.955729 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c1f1d-747b-41b2-9635-6dd397458920 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:02.956203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c1f1d-747b-41b2-9635-6dd397458920 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:02.956680 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c1f1d-747b-41b2-9635-6dd397458920 None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:02.960745 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c1f1d-747b-41b2-9635-6dd397458920 None admin] RBAC: Authorizing `identity:get_project(project_id=c57cd5c32d1447d3968cda9fc409de3c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:02.963917 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c1f1d-747b-41b2-9635-6dd397458920 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:02.967186 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 121/237] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:02 2022] GET /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c => generated 302 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:02.980362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c1f1d-747b-41b2-9635-6dd397458920 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:03.052721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ead82-4280-469d-aaae-71531b2c7512 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:03.102754 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f1c04-73e1-49a0-ae02-8ee7851d5c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:03.103685 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f1c04-73e1-49a0-ae02-8ee7851d5c11 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:03.104119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f1c04-73e1-49a0-ae02-8ee7851d5c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:03.104533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f1c04-73e1-49a0-ae02-8ee7851d5c11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:03.106116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f1c04-73e1-49a0-ae02-8ee7851d5c11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:03.107941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f1c04-73e1-49a0-ae02-8ee7851d5c11 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:03.110422 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 117/238] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:03.117224 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f1c04-73e1-49a0-ae02-8ee7851d5c11 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:03.169024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:03.248146 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c1f1d-747b-41b2-9635-6dd397458920 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:03.307856 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f1c04-73e1-49a0-ae02-8ee7851d5c11 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:03.324396 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f4248-9408-478b-8648-edb730f66bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:03.325830 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f4248-9408-478b-8648-edb730f66bbe None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:03.326493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f4248-9408-478b-8648-edb730f66bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:03.327172 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f4248-9408-478b-8648-edb730f66bbe None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c/users/cdc776676bb5450d92bf017aec1a3442/roles/bad9ff35005240e99bedef3fddb1dc77` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:03.331054 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f4248-9408-478b-8648-edb730f66bbe None admin] RBAC: Authorizing `identity:create_grant(project_id=c57cd5c32d1447d3968cda9fc409de3c, user_id=cdc776676bb5450d92bf017aec1a3442, role_id=bad9ff35005240e99bedef3fddb1dc77)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:03.333027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f4248-9408-478b-8648-edb730f66bbe None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:03.368272 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 122/239] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:00:02 2022] PUT /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c/users/cdc776676bb5450d92bf017aec1a3442/roles/bad9ff35005240e99bedef3fddb1dc77 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:00:03.376463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f4248-9408-478b-8648-edb730f66bbe None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:03.414633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4c521-1e0d-4daf-8098-7dc0b7368ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:03.415749 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4c521-1e0d-4daf-8098-7dc0b7368ca8 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:03.416259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4c521-1e0d-4daf-8098-7dc0b7368ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:03.416748 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4c521-1e0d-4daf-8098-7dc0b7368ca8 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:03.417944 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4c521-1e0d-4daf-8098-7dc0b7368ca8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:03.423255 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4c521-1e0d-4daf-8098-7dc0b7368ca8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:03.492572 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:03.498925 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 118/240] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 6 16:00:03 2022] POST /identity/v3/users => generated 293 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:00:03.579593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783da089-5722-4f7a-9ef9-1478ba7d8564 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:03.580086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783da089-5722-4f7a-9ef9-1478ba7d8564 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:03.580563 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783da089-5722-4f7a-9ef9-1478ba7d8564 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:03.584286 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 119/241] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:03.594631 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ed6f5-2b6a-4c70-9fcb-c3845e005d45 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:03.595222 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ed6f5-2b6a-4c70-9fcb-c3845e005d45 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:03.595716 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ed6f5-2b6a-4c70-9fcb-c3845e005d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:03.649040 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-067ed6f5-2b6a-4c70-9fcb-c3845e005d45 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:03.652332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f4248-9408-478b-8648-edb730f66bbe None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:03.655654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-067ed6f5-2b6a-4c70-9fcb-c3845e005d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:03.717470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9592cf7-04f5-46b3-bc15-7699464c1167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:03.718517 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9592cf7-04f5-46b3-bc15-7699464c1167 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:03.718982 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9592cf7-04f5-46b3-bc15-7699464c1167 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:03.719541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9592cf7-04f5-46b3-bc15-7699464c1167 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:03.737165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9592cf7-04f5-46b3-bc15-7699464c1167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:03.743217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-e9592cf7-04f5-46b3-bc15-7699464c1167 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:00:03.743217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:03.743217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:03.743217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:03.743217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:03.743217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:03.743217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:03.743217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:03.743217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:03.743217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:03.743217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:03.743217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:03.743217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:03.743217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:03.743217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:03.743217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:03.744268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:03.744268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:03.744268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:03.744268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:03.744268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:03.744268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:03.744268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:03.744268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:03.744268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:03.744268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:03.744268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:03.744268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:03.744268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:03.744268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:03.744268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:03.744268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:03.745173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:03.745173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:03.745173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:03.745173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:03.745173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:03.745173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:03.745173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:03.745173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:03.745173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:03.745173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:03.745173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:00:03.745173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:03.747612 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 123/242] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:03.755699 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9592cf7-04f5-46b3-bc15-7699464c1167 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:03.805083 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:03.822653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 120/243] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:00:03.900654 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:03.924309 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9592cf7-04f5-46b3-bc15-7699464c1167 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:04.004305 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e26b6-745e-4664-a24a-bbc60bb37e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:04.005396 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e26b6-745e-4664-a24a-bbc60bb37e16 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:04.005922 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e26b6-745e-4664-a24a-bbc60bb37e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:04.006325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e26b6-745e-4664-a24a-bbc60bb37e16 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:04.008267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e26b6-745e-4664-a24a-bbc60bb37e16 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:04.008786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e26b6-745e-4664-a24a-bbc60bb37e16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:04.011740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e26b6-745e-4664-a24a-bbc60bb37e16 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:04.029211 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 124/244] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 16:00:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:04.038160 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ed6f5-2b6a-4c70-9fcb-c3845e005d45 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:04.126457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e26b6-745e-4664-a24a-bbc60bb37e16 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:04.217410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ed6f5-2b6a-4c70-9fcb-c3845e005d45 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:04.321099 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6db09d-4b10-4278-a845-f2724d1a7af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:04.323529 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6db09d-4b10-4278-a845-f2724d1a7af5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:04.324441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6db09d-4b10-4278-a845-f2724d1a7af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:04.325270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6db09d-4b10-4278-a845-f2724d1a7af5 None admin] PATH_INFO: `/v3/users/d8675a4b026c42049384ed8ed46cd28b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:04.327303 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e26b6-745e-4664-a24a-bbc60bb37e16 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:04.363830 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6db09d-4b10-4278-a845-f2724d1a7af5 None admin] RBAC: Authorizing `identity:get_user(user_id=d8675a4b026c42049384ed8ed46cd28b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:04.367242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6db09d-4b10-4278-a845-f2724d1a7af5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:04.370546 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 121/245] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:00:04 2022] GET /identity/v3/users/d8675a4b026c42049384ed8ed46cd28b => generated 297 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:04.374779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ea7d2-4835-44fe-ac27-f4faa21efac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:04.374779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ea7d2-4835-44fe-ac27-f4faa21efac5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:04.374779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ea7d2-4835-44fe-ac27-f4faa21efac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:04.374779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ea7d2-4835-44fe-ac27-f4faa21efac5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:04.380912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6db09d-4b10-4278-a845-f2724d1a7af5 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:04.396854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ea7d2-4835-44fe-ac27-f4faa21efac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:04.403405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-327ea7d2-4835-44fe-ac27-f4faa21efac5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:04.403405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:04.403405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:04.403405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:04.403405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:04.403405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:04.403405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:04.403405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:04.403405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:04.403405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:04.403405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:04.403405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:04.403405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:04.403405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:04.403405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:04.403405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:04.405337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:04.405337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:04.405337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:04.405337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:04.405337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:04.405337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:04.405337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:04.405337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:04.405337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:04.405337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:04.405337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:04.405337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:04.405337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:04.405337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:04.405337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:04.405337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:04.406958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:04.406958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:04.406958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:04.406958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:04.406958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:04.406958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:04.406958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:04.406958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:04.406958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:04.406958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:04.406958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:04.406958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:04.406958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 125/246] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:04.411520 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ea7d2-4835-44fe-ac27-f4faa21efac5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:04.430802 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:04.533643 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:04.559803 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ea7d2-4835-44fe-ac27-f4faa21efac5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:04.649196 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6db09d-4b10-4278-a845-f2724d1a7af5 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:04.655774 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227eed33-7a5b-45dd-a60d-a07122593438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:04.656839 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227eed33-7a5b-45dd-a60d-a07122593438 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:04.657325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227eed33-7a5b-45dd-a60d-a07122593438 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:04.657788 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227eed33-7a5b-45dd-a60d-a07122593438 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:04.659051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227eed33-7a5b-45dd-a60d-a07122593438 None admin] RBAC: Adding query filter params (name=member) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:04.659568 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227eed33-7a5b-45dd-a60d-a07122593438 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:04.662221 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227eed33-7a5b-45dd-a60d-a07122593438 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:04.699798 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 126/247] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 16:00:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:04.705173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227eed33-7a5b-45dd-a60d-a07122593438 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:04.753158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491a433-9b1b-426c-9b32-81038e91ee1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:04.754197 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491a433-9b1b-426c-9b32-81038e91ee1e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:04.754698 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491a433-9b1b-426c-9b32-81038e91ee1e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:04.755183 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491a433-9b1b-426c-9b32-81038e91ee1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:04.924304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491a433-9b1b-426c-9b32-81038e91ee1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:04.925975 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491a433-9b1b-426c-9b32-81038e91ee1e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:04.976115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:05.012987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:05.019080 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227eed33-7a5b-45dd-a60d-a07122593438 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:05.108589 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 122/248] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 729 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 16:00:05.114862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491a433-9b1b-426c-9b32-81038e91ee1e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:05.139452 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbbab08-d639-4ea8-a9ec-5365706aa7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:05.140134 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbab08-d639-4ea8-a9ec-5365706aa7d5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:05.140420 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbab08-d639-4ea8-a9ec-5365706aa7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:05.141275 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbab08-d639-4ea8-a9ec-5365706aa7d5 None admin] PATH_INFO: `/v3/users/affc11afd71f4f4687e2259f7362c0fe` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:05.142320 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbbab08-d639-4ea8-a9ec-5365706aa7d5 None admin] RBAC: Authorizing `identity:get_user(user_id=affc11afd71f4f4687e2259f7362c0fe)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:05.143878 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbbab08-d639-4ea8-a9ec-5365706aa7d5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:05.145923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 127/249] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:00:04 2022] GET /identity/v3/users/affc11afd71f4f4687e2259f7362c0fe => generated 297 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:05.151895 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbbab08-d639-4ea8-a9ec-5365706aa7d5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:05.376717 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491a433-9b1b-426c-9b32-81038e91ee1e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:05.465661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fff1ec-8d8b-4b54-8f1f-d2d2485573b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:05.466311 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fff1ec-8d8b-4b54-8f1f-d2d2485573b2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:05.466588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fff1ec-8d8b-4b54-8f1f-d2d2485573b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:05.466888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fff1ec-8d8b-4b54-8f1f-d2d2485573b2 None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:05.468262 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fff1ec-8d8b-4b54-8f1f-d2d2485573b2 None admin] RBAC: Authorizing `identity:get_project(project_id=c57cd5c32d1447d3968cda9fc409de3c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:05.469764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fff1ec-8d8b-4b54-8f1f-d2d2485573b2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:05.476828 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 123/250] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:05 2022] GET /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c => generated 302 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:05.478140 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbed7b-fe92-4f24-931a-ee81cdc7155e None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:05.478424 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbed7b-fe92-4f24-931a-ee81cdc7155e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:05.478802 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbed7b-fe92-4f24-931a-ee81cdc7155e None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:05.480349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 124/251] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:05.485342 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbed7b-fe92-4f24-931a-ee81cdc7155e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:05.491738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbbab08-d639-4ea8-a9ec-5365706aa7d5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:05.557876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:05.560263 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74f485-295c-480b-80b9-59f02f77cf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:05.561905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74f485-295c-480b-80b9-59f02f77cf9b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:05.562695 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74f485-295c-480b-80b9-59f02f77cf9b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:05.563498 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74f485-295c-480b-80b9-59f02f77cf9b None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:05.566005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74f485-295c-480b-80b9-59f02f77cf9b None admin] RBAC: Authorizing `identity:get_project(project_id=c57cd5c32d1447d3968cda9fc409de3c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:05.569319 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74f485-295c-480b-80b9-59f02f77cf9b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:05.572945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 128/252] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:05 2022] GET /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c => generated 302 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:05.575654 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a26df4-3b7d-4f47-8c03-7e942333d1a6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:05.576471 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a26df4-3b7d-4f47-8c03-7e942333d1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:05.577237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a26df4-3b7d-4f47-8c03-7e942333d1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:05.615375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-92a26df4-3b7d-4f47-8c03-7e942333d1a6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:05.620623 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-92a26df4-3b7d-4f47-8c03-7e942333d1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:05.647259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:05.756736 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbed7b-fe92-4f24-931a-ee81cdc7155e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:05.761448 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 129/253] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:05.763470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a26df4-3b7d-4f47-8c03-7e942333d1a6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:05.837104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13ba19-23e8-4744-9b06-4014acced552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:05.837383 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13ba19-23e8-4744-9b06-4014acced552 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:05.837743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13ba19-23e8-4744-9b06-4014acced552 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:05.837891 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13ba19-23e8-4744-9b06-4014acced552 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:05.838645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13ba19-23e8-4744-9b06-4014acced552 None admin] RBAC: Adding query filter params (role.id=11b32861fcd8469382d1a7047a95ea70, scope.project.id=c57cd5c32d1447d3968cda9fc409de3c, user.id=d8675a4b026c42049384ed8ed46cd28b) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:05.838991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13ba19-23e8-4744-9b06-4014acced552 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:05.840440 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13ba19-23e8-4744-9b06-4014acced552 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:05.854290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 125/254] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:00:05 2022] GET /identity/v3/role_assignments?user.id=d8675a4b026c42049384ed8ed46cd28b&scope.project.id=c57cd5c32d1447d3968cda9fc409de3c&role.id=11b32861fcd8469382d1a7047a95ea70 => generated 259 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:05.971911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a26df4-3b7d-4f47-8c03-7e942333d1a6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:06.043423 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8cbae5-5077-49b4-b1d6-965a07487147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:06.045050 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8cbae5-5077-49b4-b1d6-965a07487147 None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:06.045798 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8cbae5-5077-49b4-b1d6-965a07487147 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:06.046511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8cbae5-5077-49b4-b1d6-965a07487147 None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c/users/affc11afd71f4f4687e2259f7362c0fe/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:06.050587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8cbae5-5077-49b4-b1d6-965a07487147 None admin] RBAC: Authorizing `identity:create_grant(project_id=c57cd5c32d1447d3968cda9fc409de3c, user_id=affc11afd71f4f4687e2259f7362c0fe, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:06.055954 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8cbae5-5077-49b4-b1d6-965a07487147 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:06.088206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 130/255] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:00:05 2022] PUT /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c/users/affc11afd71f4f4687e2259f7362c0fe/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:00:06.318729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13ba19-23e8-4744-9b06-4014acced552 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:06.447991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:06.467965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13ba19-23e8-4744-9b06-4014acced552 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:06.510813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f9a57-1740-43af-87b8-41985fd22f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:06.511469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f9a57-1740-43af-87b8-41985fd22f99 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:06.511751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f9a57-1740-43af-87b8-41985fd22f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:06.512032 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f9a57-1740-43af-87b8-41985fd22f99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:06.528502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f9a57-1740-43af-87b8-41985fd22f99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:06.534056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-d47f9a57-1740-43af-87b8-41985fd22f99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:00:06.534056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:06.534056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:06.534056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:06.534056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:06.534056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:06.534056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:06.534056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:06.534056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:06.534056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:06.534056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:06.534056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:06.534056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:06.534056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:06.534056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:06.534056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:06.535002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:06.535002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:06.535002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:06.535002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:06.535002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:06.535002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:06.535002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:06.535002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:06.535002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:06.535002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:06.535002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:06.535002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:06.535002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:06.535002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:06.535002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:06.535002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:06.535781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:06.535781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:06.535781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:06.535781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:06.535781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:06.535781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:06.535781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:06.535781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:06.535781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:06.535781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:06.535781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:00:06.535781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:06.535781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 126/256] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:00:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:00:06.542571 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8cbae5-5077-49b4-b1d6-965a07487147 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:06.595073 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:06.793214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8cbae5-5077-49b4-b1d6-965a07487147 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:06.852204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d3e01-681f-4db6-a7d6-16bab3c8125e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:06.852756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d3e01-681f-4db6-a7d6-16bab3c8125e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:06.853037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d3e01-681f-4db6-a7d6-16bab3c8125e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:06.853313 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d3e01-681f-4db6-a7d6-16bab3c8125e None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:06.853996 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d3e01-681f-4db6-a7d6-16bab3c8125e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:06.854308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d3e01-681f-4db6-a7d6-16bab3c8125e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:06.855782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d3e01-681f-4db6-a7d6-16bab3c8125e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:06.871087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 131/257] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:06.877500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f9a57-1740-43af-87b8-41985fd22f99 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:07.015489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f9a57-1740-43af-87b8-41985fd22f99 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:07.061825 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1181c-9b9e-4893-8e80-18c1be1d797e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:07.062481 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1181c-9b9e-4893-8e80-18c1be1d797e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:07.062787 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1181c-9b9e-4893-8e80-18c1be1d797e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:07.063065 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1181c-9b9e-4893-8e80-18c1be1d797e None admin] PATH_INFO: `/v3/users/cdc776676bb5450d92bf017aec1a3442` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:07.067402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c1181c-9b9e-4893-8e80-18c1be1d797e None admin] RBAC: Authorizing `identity:get_user(user_id=cdc776676bb5450d92bf017aec1a3442)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:07.068992 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c1181c-9b9e-4893-8e80-18c1be1d797e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:07.071119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 127/258] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:00:06 2022] GET /identity/v3/users/cdc776676bb5450d92bf017aec1a3442 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:07.076929 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d3e01-681f-4db6-a7d6-16bab3c8125e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:07.259169 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:07.336938 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd49025-42e2-4d5c-920f-e12272ba827b None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:07.337580 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd49025-42e2-4d5c-920f-e12272ba827b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:07.338079 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd49025-42e2-4d5c-920f-e12272ba827b None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:07.340180 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 128/259] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:07.345851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2163e8-aa61-4ee9-b7b6-f7181f937817 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:07.346446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2163e8-aa61-4ee9-b7b6-f7181f937817 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:07.346980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2163e8-aa61-4ee9-b7b6-f7181f937817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:07.353684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d3e01-681f-4db6-a7d6-16bab3c8125e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:07.368682 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-5d2163e8-aa61-4ee9-b7b6-f7181f937817 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:07.370262 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5d2163e8-aa61-4ee9-b7b6-f7181f937817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:07.387444 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:07.462226 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 129/260] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:00:07.470699 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e95c41-5d08-405d-a22a-0bc4a5416fa9 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:07.471802 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e95c41-5d08-405d-a22a-0bc4a5416fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:07.472288 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e95c41-5d08-405d-a22a-0bc4a5416fa9 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:07.473981 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 130/261] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:07 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:07.507753 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408b5dd-cdc7-4834-84b1-3c164c3b1038 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:07.508614 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd41bf-ac85-439d-8456-1d1d3faa5aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:07.508999 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408b5dd-cdc7-4834-84b1-3c164c3b1038 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:07.509769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408b5dd-cdc7-4834-84b1-3c164c3b1038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:07.509926 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd41bf-ac85-439d-8456-1d1d3faa5aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:07.510261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd41bf-ac85-439d-8456-1d1d3faa5aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:07.510620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd41bf-ac85-439d-8456-1d1d3faa5aa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:07.589175 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-3408b5dd-cdc7-4834-84b1-3c164c3b1038 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:07.591364 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3408b5dd-cdc7-4834-84b1-3c164c3b1038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:07.749719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd41bf-ac85-439d-8456-1d1d3faa5aa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:07.752578 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd41bf-ac85-439d-8456-1d1d3faa5aa6 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:07.814386 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 131/262] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:07.816643 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d2bad-e712-4b6c-b03d-946848242187 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:07.817143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d2bad-e712-4b6c-b03d-946848242187 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:07.817629 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d2bad-e712-4b6c-b03d-946848242187 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:07.821601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 132/263] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:07.840468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea33ef-30d9-4999-94a7-6d837345c2a8 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:07.841054 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea33ef-30d9-4999-94a7-6d837345c2a8 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:07.841548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea33ef-30d9-4999-94a7-6d837345c2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:07.896308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-76ea33ef-30d9-4999-94a7-6d837345c2a8 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:07.898063 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-76ea33ef-30d9-4999-94a7-6d837345c2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:08.012656 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:08.049056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 133/264] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:08.142627 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 132/265] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1066 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:00:08.150013 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea33ef-30d9-4999-94a7-6d837345c2a8 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:08.169768 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:08.387990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd41bf-ac85-439d-8456-1d1d3faa5aa6 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:08.421795 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea33ef-30d9-4999-94a7-6d837345c2a8 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:08.532439 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf596c25-cb61-4806-91d6-2c61c9fa402a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:08.533836 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf596c25-cb61-4806-91d6-2c61c9fa402a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:08.534544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf596c25-cb61-4806-91d6-2c61c9fa402a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:08.535060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf596c25-cb61-4806-91d6-2c61c9fa402a None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:08.540159 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf596c25-cb61-4806-91d6-2c61c9fa402a None admin] RBAC: Authorizing `identity:get_project(project_id=c57cd5c32d1447d3968cda9fc409de3c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:08.543879 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf596c25-cb61-4806-91d6-2c61c9fa402a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:08.549349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 134/266] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:08 2022] GET /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c => generated 302 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:08.551425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf596c25-cb61-4806-91d6-2c61c9fa402a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:08.582470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd41bf-ac85-439d-8456-1d1d3faa5aa6 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:08.654769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e14bd-0239-4bfa-8cb1-c323e3e73690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:08.656028 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e14bd-0239-4bfa-8cb1-c323e3e73690 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:08.656532 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e14bd-0239-4bfa-8cb1-c323e3e73690 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:08.657268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e14bd-0239-4bfa-8cb1-c323e3e73690 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:08.672810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:08.684936 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e14bd-0239-4bfa-8cb1-c323e3e73690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:08.697763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-ab7e14bd-0239-4bfa-8cb1-c323e3e73690 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:08.697763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:08.697763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:08.697763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:08.697763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:08.697763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:08.697763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:08.697763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:08.697763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:08.697763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:08.697763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:08.697763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:08.697763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:08.697763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:08.697763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:08.697763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:08.699598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:08.699598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:08.699598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:08.699598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:08.699598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:08.699598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:08.699598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:08.699598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:08.699598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:08.699598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:08.699598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:08.699598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:08.699598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:08.699598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:08.699598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:08.699598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:08.701108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:08.701108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:08.701108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:08.701108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:08.701108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:08.701108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:08.701108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:08.701108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:08.701108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:08.701108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:08.701108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:08.701108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:08.703372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf596c25-cb61-4806-91d6-2c61c9fa402a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:08.706203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 133/267] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:00:08.708104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e14bd-0239-4bfa-8cb1-c323e3e73690 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:08.744244 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79081a-7586-444f-98f8-aa8f8825fbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:08.745008 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79081a-7586-444f-98f8-aa8f8825fbcc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:08.745346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79081a-7586-444f-98f8-aa8f8825fbcc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:08.745686 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79081a-7586-444f-98f8-aa8f8825fbcc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:08.758871 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79081a-7586-444f-98f8-aa8f8825fbcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:08.764821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-1f79081a-7586-444f-98f8-aa8f8825fbcc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:00:08.764821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:08.764821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:08.764821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:08.764821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:08.764821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:08.764821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:08.764821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:08.764821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:08.764821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:08.764821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:08.764821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:08.764821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:08.764821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:08.764821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:08.764821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:08.766535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:08.766535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:08.766535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:08.766535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:08.766535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:08.766535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:08.766535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:08.766535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:08.766535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:08.766535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:08.766535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:08.766535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:08.766535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:08.766535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:08.766535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:08.766535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:08.768126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:08.768126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:08.768126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:08.768126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:08.768126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:08.768126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:08.768126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:08.768126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:08.768126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:08.768126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:08.768126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:00:08.768126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:08.768126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 135/268] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 16:00:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:08.768126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79081a-7586-444f-98f8-aa8f8825fbcc None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:08.782698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:08.895129 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79081a-7586-444f-98f8-aa8f8825fbcc None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:08.927889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cfb336-6da6-4334-ab0c-c7daadfb5434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:08.929032 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfb336-6da6-4334-ab0c-c7daadfb5434 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:08.929315 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfb336-6da6-4334-ab0c-c7daadfb5434 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:08.929586 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfb336-6da6-4334-ab0c-c7daadfb5434 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:08.930504 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cfb336-6da6-4334-ab0c-c7daadfb5434 None admin] RBAC: Adding query filter params (role.id=ba05677650e046a48d5f19440d48fbf2, scope.project.id=c57cd5c32d1447d3968cda9fc409de3c, user.id=cdc776676bb5450d92bf017aec1a3442) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:08.930844 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cfb336-6da6-4334-ab0c-c7daadfb5434 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:08.932353 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cfb336-6da6-4334-ab0c-c7daadfb5434 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:08.934541 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:08.947092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 136/269] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:00:08 2022] GET /identity/v3/role_assignments?user.id=cdc776676bb5450d92bf017aec1a3442&scope.project.id=c57cd5c32d1447d3968cda9fc409de3c&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:08.948747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cfb336-6da6-4334-ab0c-c7daadfb5434 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:08.955624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e14bd-0239-4bfa-8cb1-c323e3e73690 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:09.045468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffdf1f-c406-4472-8a4c-2cf7b7b9d387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:09.047958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffdf1f-c406-4472-8a4c-2cf7b7b9d387 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:09.048480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffdf1f-c406-4472-8a4c-2cf7b7b9d387 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:09.049154 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffdf1f-c406-4472-8a4c-2cf7b7b9d387 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:09.053793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffdf1f-c406-4472-8a4c-2cf7b7b9d387 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:09.056873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffdf1f-c406-4472-8a4c-2cf7b7b9d387 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:09.061423 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 134/270] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:09.062811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffdf1f-c406-4472-8a4c-2cf7b7b9d387 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:09.098800 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cfb336-6da6-4334-ab0c-c7daadfb5434 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:09.141043 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d593f-4d88-4262-b6b3-a884c48d3b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:09.141812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d593f-4d88-4262-b6b3-a884c48d3b63 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:09.142095 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d593f-4d88-4262-b6b3-a884c48d3b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:09.142365 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d593f-4d88-4262-b6b3-a884c48d3b63 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:09.143096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d593f-4d88-4262-b6b3-a884c48d3b63 None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:09.143401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d593f-4d88-4262-b6b3-a884c48d3b63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:09.144816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d593f-4d88-4262-b6b3-a884c48d3b63 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:09.155106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 137/271] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 16:00:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:09.156531 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e64f5f-d7fa-453b-9ee1-ca0eb85543ed None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:09.156820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e64f5f-d7fa-453b-9ee1-ca0eb85543ed None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:09.157096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e64f5f-d7fa-453b-9ee1-ca0eb85543ed None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:09.158772 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 138/272] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:09.159690 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e64f5f-d7fa-453b-9ee1-ca0eb85543ed None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:09.264364 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:09.266177 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:09.283443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffdf1f-c406-4472-8a4c-2cf7b7b9d387 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:09.316923 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e64f5f-d7fa-453b-9ee1-ca0eb85543ed None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:09.351655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d8a2e-eefb-4d4a-ad1c-8dcc4a56257e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:09.352585 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d8a2e-eefb-4d4a-ad1c-8dcc4a56257e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:09.352870 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d8a2e-eefb-4d4a-ad1c-8dcc4a56257e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:09.353256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d8a2e-eefb-4d4a-ad1c-8dcc4a56257e None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:09.354486 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d8a2e-eefb-4d4a-ad1c-8dcc4a56257e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:09.355033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d8a2e-eefb-4d4a-ad1c-8dcc4a56257e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:09.357590 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d8a2e-eefb-4d4a-ad1c-8dcc4a56257e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:09.358726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c34866-74c9-4088-b5eb-25f57d994b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:09.359541 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c34866-74c9-4088-b5eb-25f57d994b7d None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:09.360014 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c34866-74c9-4088-b5eb-25f57d994b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:09.360279 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c34866-74c9-4088-b5eb-25f57d994b7d None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:09.360948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c34866-74c9-4088-b5eb-25f57d994b7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:09.362439 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c34866-74c9-4088-b5eb-25f57d994b7d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:09.378638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 135/273] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:00:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:09.378638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d8a2e-eefb-4d4a-ad1c-8dcc4a56257e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:09.399817 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 139/274] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 6 16:00:09 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:00:09.401463 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a885ab-447b-4b0b-a8b8-ca74a6442707 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:09.402170 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a885ab-447b-4b0b-a8b8-ca74a6442707 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:09.402170 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a885ab-447b-4b0b-a8b8-ca74a6442707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:09.435119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-40a885ab-447b-4b0b-a8b8-ca74a6442707 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:09.437114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-40a885ab-447b-4b0b-a8b8-ca74a6442707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:09.557770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d8a2e-eefb-4d4a-ad1c-8dcc4a56257e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:09.577837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 140/275] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:09.579385 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a885ab-447b-4b0b-a8b8-ca74a6442707 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:09.617998 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a530ba-f3b1-4be9-b50d-b96d2b13e00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:09.618873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a530ba-f3b1-4be9-b50d-b96d2b13e00a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:09.619573 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a530ba-f3b1-4be9-b50d-b96d2b13e00a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:09.620295 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a530ba-f3b1-4be9-b50d-b96d2b13e00a None admin] PATH_INFO: `/v3/users/9f080f517c1a41e8b078196fae76a1dc` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:09.640680 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a530ba-f3b1-4be9-b50d-b96d2b13e00a None admin] RBAC: Authorizing `identity:get_user(user_id=9f080f517c1a41e8b078196fae76a1dc)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:09.642252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a530ba-f3b1-4be9-b50d-b96d2b13e00a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:09.644536 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 136/276] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:00:09 2022] GET /identity/v3/users/9f080f517c1a41e8b078196fae76a1dc => generated 293 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:09.645906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5822d-2a28-4e9c-b0f8-5aae04000f3b None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:09.646199 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5822d-2a28-4e9c-b0f8-5aae04000f3b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:09.646477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5822d-2a28-4e9c-b0f8-5aae04000f3b None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:09.652444 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 137/277] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:09.657957 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bba8e-d029-48de-95f9-a4b6a8315260 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:09.658235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bba8e-d029-48de-95f9-a4b6a8315260 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:09.658688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bba8e-d029-48de-95f9-a4b6a8315260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:09.664580 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:09.682030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-041bba8e-d029-48de-95f9-a4b6a8315260 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:09.683564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-041bba8e-d029-48de-95f9-a4b6a8315260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:09.693662 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:09.771122 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 138/278] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:00:09.773106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041bba8e-d029-48de-95f9-a4b6a8315260 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:09.777970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a885ab-447b-4b0b-a8b8-ca74a6442707 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:09.868400 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725aa456-5b73-4c41-8299-4f5abea304c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:09.870018 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725aa456-5b73-4c41-8299-4f5abea304c5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:09.870877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725aa456-5b73-4c41-8299-4f5abea304c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:09.871543 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725aa456-5b73-4c41-8299-4f5abea304c5 None admin] PATH_INFO: `/v3/users/cdc776676bb5450d92bf017aec1a3442` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:09.874045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725aa456-5b73-4c41-8299-4f5abea304c5 None admin] RBAC: Authorizing `identity:get_user(user_id=cdc776676bb5450d92bf017aec1a3442)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:09.877325 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725aa456-5b73-4c41-8299-4f5abea304c5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:09.881095 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 141/279] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:00:09 2022] GET /identity/v3/users/cdc776676bb5450d92bf017aec1a3442 => generated 283 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:09.888215 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725aa456-5b73-4c41-8299-4f5abea304c5 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:09.906242 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041bba8e-d029-48de-95f9-a4b6a8315260 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:09.954475 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a7d55-bd65-4151-b407-9e9605be4409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:09.955148 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a7d55-bd65-4151-b407-9e9605be4409 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:09.955426 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a7d55-bd65-4151-b407-9e9605be4409 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:09.955700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a7d55-bd65-4151-b407-9e9605be4409 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:09.956472 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a7d55-bd65-4151-b407-9e9605be4409 None admin] RBAC: Adding query filter params (role.id=e89162686187404da84651e0a3632b7b, user.id=9f080f517c1a41e8b078196fae76a1dc, scope.system=all) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:09.956785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a7d55-bd65-4151-b407-9e9605be4409 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:09.960670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a7d55-bd65-4151-b407-9e9605be4409 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:09.974173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 139/280] 10.222.0.26 () {56 vars in 1308 bytes} [Tue Sep 6 16:00:09 2022] GET /identity/v3/role_assignments?user.id=9f080f517c1a41e8b078196fae76a1dc&scope.system=all&role.id=e89162686187404da84651e0a3632b7b => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:09.975714 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a7d55-bd65-4151-b407-9e9605be4409 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:09.998401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:10.203304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725aa456-5b73-4c41-8299-4f5abea304c5 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:10.242340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:10.281267 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4d8c1-1258-45ea-a38d-4350fd418fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:10.282328 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4d8c1-1258-45ea-a38d-4350fd418fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:10.282916 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4d8c1-1258-45ea-a38d-4350fd418fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:10.283404 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4d8c1-1258-45ea-a38d-4350fd418fc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:10.325134 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a7d55-bd65-4151-b407-9e9605be4409 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:10.426820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d4d8c1-1258-45ea-a38d-4350fd418fc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:10.427455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fd25e-c08e-4db3-be7f-c7882e3e5803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:10.428313 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fd25e-c08e-4db3-be7f-c7882e3e5803 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:10.428659 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fd25e-c08e-4db3-be7f-c7882e3e5803 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:10.428954 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fd25e-c08e-4db3-be7f-c7882e3e5803 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:10.429611 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d4d8c1-1258-45ea-a38d-4350fd418fc1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:10.469073 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8fd25e-c08e-4db3-be7f-c7882e3e5803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:10.479534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-1e8fd25e-c08e-4db3-be7f-c7882e3e5803 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:10.479534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:10.479534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:10.479534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:10.479534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:10.479534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:10.479534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:10.479534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:10.479534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:10.479534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:10.479534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:10.479534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:10.479534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:10.479534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:10.479534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:10.479534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:10.481116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:10.481116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:10.481116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:10.481116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:10.481116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:10.481116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:10.481116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:10.481116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:10.481116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:10.481116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:10.481116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:10.481116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:10.481116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:10.481116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:10.481116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:10.481116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:10.482550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:10.482550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:10.482550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:10.482550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:10.482550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:10.482550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:10.482550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:10.482550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:10.482550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:10.482550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:10.482550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:10.482550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:10.486550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 140/281] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 512 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:10.492942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fd25e-c08e-4db3-be7f-c7882e3e5803 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:10.675622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:10.727128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 142/282] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 840 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:00:10.727988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4d8c1-1258-45ea-a38d-4350fd418fc1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:10.755339 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fd25e-c08e-4db3-be7f-c7882e3e5803 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:10.773921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:10.847228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f91ca-c12a-4ed6-8a5b-dddd201d76a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:10.848300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f91ca-c12a-4ed6-8a5b-dddd201d76a4 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:10.848761 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f91ca-c12a-4ed6-8a5b-dddd201d76a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:10.849211 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f91ca-c12a-4ed6-8a5b-dddd201d76a4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:10.888879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f91ca-c12a-4ed6-8a5b-dddd201d76a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:10.901321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-3b8f91ca-c12a-4ed6-8a5b-dddd201d76a4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:00:10.901321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:10.901321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:10.901321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:10.901321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:10.901321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:10.901321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:10.901321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:10.901321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:10.901321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:10.901321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:10.901321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:10.901321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:10.901321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:10.901321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:10.901321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:10.902270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:10.902270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:10.902270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:10.902270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:10.902270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:10.902270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:10.902270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:10.902270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:10.902270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:10.902270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:10.902270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:10.902270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:10.902270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:10.902270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:10.902270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:10.902270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:10.903233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:10.903233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:10.903233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:10.903233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:10.903233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:10.903233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:10.903233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:10.903233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:10.903233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:10.903233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:10.903233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:00:10.903233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:10.904363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 141/283] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:10.905287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f91ca-c12a-4ed6-8a5b-dddd201d76a4 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:11.034389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4d8c1-1258-45ea-a38d-4350fd418fc1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:11.151478 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d075ff-d27c-4eed-8abe-c331011d541c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:11.152105 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d075ff-d27c-4eed-8abe-c331011d541c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:11.152556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d075ff-d27c-4eed-8abe-c331011d541c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:11.152983 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d075ff-d27c-4eed-8abe-c331011d541c None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:11.153821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d075ff-d27c-4eed-8abe-c331011d541c None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:11.154304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d075ff-d27c-4eed-8abe-c331011d541c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:11.155899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d075ff-d27c-4eed-8abe-c331011d541c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:11.166321 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 143/284] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 16:00:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:11.167547 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d075ff-d27c-4eed-8abe-c331011d541c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:11.185719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f91ca-c12a-4ed6-8a5b-dddd201d76a4 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:11.256374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:11.281402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d075ff-d27c-4eed-8abe-c331011d541c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:11.285641 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:11.321884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c596201-afa3-4285-93c8-b1278fd5acff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:11.323437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c596201-afa3-4285-93c8-b1278fd5acff None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:11.325222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c596201-afa3-4285-93c8-b1278fd5acff None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:11.326099 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1461954-d938-4281-9bc6-7cd6e28001df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:11.326610 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c596201-afa3-4285-93c8-b1278fd5acff None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:11.328062 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1461954-d938-4281-9bc6-7cd6e28001df None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:11.328951 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c596201-afa3-4285-93c8-b1278fd5acff None admin] RBAC: Authorizing `identity:get_project(project_id=c57cd5c32d1447d3968cda9fc409de3c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:11.329502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1461954-d938-4281-9bc6-7cd6e28001df None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:11.331223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c596201-afa3-4285-93c8-b1278fd5acff None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:11.331726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1461954-d938-4281-9bc6-7cd6e28001df None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:11.332655 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1461954-d938-4281-9bc6-7cd6e28001df None admin] RBAC: Adding query filter params (name=reader) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:11.333161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1461954-d938-4281-9bc6-7cd6e28001df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:11.335297 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1461954-d938-4281-9bc6-7cd6e28001df None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:11.337540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 142/285] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:10 2022] GET /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c => generated 302 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:11.343687 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c596201-afa3-4285-93c8-b1278fd5acff None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:11.378573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 144/286] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 16:00:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:11.380065 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1461954-d938-4281-9bc6-7cd6e28001df None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:11.468202 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c596201-afa3-4285-93c8-b1278fd5acff None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:11.510396 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d564ef4-0c3f-4509-a5fa-d0b9987ab3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:11.511550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d564ef4-0c3f-4509-a5fa-d0b9987ab3dc None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:11.512035 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d564ef4-0c3f-4509-a5fa-d0b9987ab3dc None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:11.512510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d564ef4-0c3f-4509-a5fa-d0b9987ab3dc None admin] PATH_INFO: `/v3/users/affc11afd71f4f4687e2259f7362c0fe` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:11.514145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d564ef4-0c3f-4509-a5fa-d0b9987ab3dc None admin] RBAC: Authorizing `identity:get_user(user_id=affc11afd71f4f4687e2259f7362c0fe)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:11.515921 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d564ef4-0c3f-4509-a5fa-d0b9987ab3dc None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:11.517778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 143/287] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:00:11 2022] GET /identity/v3/users/affc11afd71f4f4687e2259f7362c0fe => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:11.518805 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d564ef4-0c3f-4509-a5fa-d0b9987ab3dc None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:11.594765 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:11.644389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d564ef4-0c3f-4509-a5fa-d0b9987ab3dc None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:11.691170 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f833ccd-4d61-4361-9495-4af1f439711f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:11.692194 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f833ccd-4d61-4361-9495-4af1f439711f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:11.692681 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f833ccd-4d61-4361-9495-4af1f439711f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:11.693477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f833ccd-4d61-4361-9495-4af1f439711f None admin] PATH_INFO: `/v3/users/d8675a4b026c42049384ed8ed46cd28b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:11.696147 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f833ccd-4d61-4361-9495-4af1f439711f None admin] RBAC: Authorizing `identity:get_user(user_id=d8675a4b026c42049384ed8ed46cd28b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:11.698168 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f833ccd-4d61-4361-9495-4af1f439711f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:11.700463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 144/288] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:00:11 2022] GET /identity/v3/users/d8675a4b026c42049384ed8ed46cd28b => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:11.701497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f833ccd-4d61-4361-9495-4af1f439711f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:11.813575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1461954-d938-4281-9bc6-7cd6e28001df None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:11.917945 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87360d8-2eb2-4aae-b39f-61b10e5605f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:11.919272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87360d8-2eb2-4aae-b39f-61b10e5605f9 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:11.925489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87360d8-2eb2-4aae-b39f-61b10e5605f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:11.925980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87360d8-2eb2-4aae-b39f-61b10e5605f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:11.927476 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87360d8-2eb2-4aae-b39f-61b10e5605f9 None admin] RBAC: Adding query filter params (role.id=bad9ff35005240e99bedef3fddb1dc77, scope.project.id=c57cd5c32d1447d3968cda9fc409de3c, user.id=cdc776676bb5450d92bf017aec1a3442) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:11.928037 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87360d8-2eb2-4aae-b39f-61b10e5605f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:11.933624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87360d8-2eb2-4aae-b39f-61b10e5605f9 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:11.953582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 145/289] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:00:11 2022] GET /identity/v3/role_assignments?user.id=cdc776676bb5450d92bf017aec1a3442&scope.project.id=c57cd5c32d1447d3968cda9fc409de3c&role.id=bad9ff35005240e99bedef3fddb1dc77 => generated 608 bytes in 574 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:11.955127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87360d8-2eb2-4aae-b39f-61b10e5605f9 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:11.971178 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:11.975803 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f833ccd-4d61-4361-9495-4af1f439711f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:12.053720 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:12.062941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc7c8c-a9e5-4fa8-8af5-0c3c0e736ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:12.063946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc7c8c-a9e5-4fa8-8af5-0c3c0e736ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:12.064436 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc7c8c-a9e5-4fa8-8af5-0c3c0e736ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:12.064947 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc7c8c-a9e5-4fa8-8af5-0c3c0e736ec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:12.209954 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc7c8c-a9e5-4fa8-8af5-0c3c0e736ec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:12.212360 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc7c8c-a9e5-4fa8-8af5-0c3c0e736ec5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:12.274357 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87360d8-2eb2-4aae-b39f-61b10e5605f9 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:12.331652 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a14c29-a6e9-4080-b3a0-92ed08799654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:12.332325 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a14c29-a6e9-4080-b3a0-92ed08799654 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:12.332608 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a14c29-a6e9-4080-b3a0-92ed08799654 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:12.333010 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a14c29-a6e9-4080-b3a0-92ed08799654 None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:12.334360 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a14c29-a6e9-4080-b3a0-92ed08799654 None admin] RBAC: Authorizing `identity:get_project(project_id=c57cd5c32d1447d3968cda9fc409de3c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:12.336662 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a14c29-a6e9-4080-b3a0-92ed08799654 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:12.338764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 146/290] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:11 2022] GET /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c => generated 302 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:12.354264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a14c29-a6e9-4080-b3a0-92ed08799654 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:12.504941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:12.564630 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 145/291] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 863 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:00:12.580683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc7c8c-a9e5-4fa8-8af5-0c3c0e736ec5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:12.594816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a14c29-a6e9-4080-b3a0-92ed08799654 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:12.691070 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:12.707833 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5a753-bea8-4847-98fe-785eefd8a564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:12.709072 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5a753-bea8-4847-98fe-785eefd8a564 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:12.709553 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5a753-bea8-4847-98fe-785eefd8a564 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:12.710009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5a753-bea8-4847-98fe-785eefd8a564 None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c/users/d8675a4b026c42049384ed8ed46cd28b/roles/11b32861fcd8469382d1a7047a95ea70` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:12.712332 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5a753-bea8-4847-98fe-785eefd8a564 None admin] RBAC: Authorizing `identity:create_grant(project_id=c57cd5c32d1447d3968cda9fc409de3c, user_id=d8675a4b026c42049384ed8ed46cd28b, role_id=11b32861fcd8469382d1a7047a95ea70)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:12.713989 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5a753-bea8-4847-98fe-785eefd8a564 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:12.745401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 147/292] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:00:12 2022] PUT /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c/users/d8675a4b026c42049384ed8ed46cd28b/roles/11b32861fcd8469382d1a7047a95ea70 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:00:12.842388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc7c8c-a9e5-4fa8-8af5-0c3c0e736ec5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:12.857644 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387c4e7-3f37-4468-a3a5-2cd85e21401d None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:12.858411 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387c4e7-3f37-4468-a3a5-2cd85e21401d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:12.858411 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387c4e7-3f37-4468-a3a5-2cd85e21401d None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:12.860010 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 148/293] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:00:12.866445 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e6ae5-7a42-4bb1-8680-6affcc2e96ff None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:12.867157 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e6ae5-7a42-4bb1-8680-6affcc2e96ff None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:12.867625 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e6ae5-7a42-4bb1-8680-6affcc2e96ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:12.904579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-042e6ae5-7a42-4bb1-8680-6affcc2e96ff None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:12.906387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-042e6ae5-7a42-4bb1-8680-6affcc2e96ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:12.924888 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b7ef3-64a2-4423-a170-4315d35d31aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:12.925964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b7ef3-64a2-4423-a170-4315d35d31aa None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:12.926431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b7ef3-64a2-4423-a170-4315d35d31aa None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:12.926914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b7ef3-64a2-4423-a170-4315d35d31aa None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:12.932348 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b7ef3-64a2-4423-a170-4315d35d31aa None admin] RBAC: Authorizing `identity:get_project(project_id=c57cd5c32d1447d3968cda9fc409de3c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:12.934994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b7ef3-64a2-4423-a170-4315d35d31aa None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:12.939330 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 146/294] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:12 2022] GET /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c => generated 302 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:12.946923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b7ef3-64a2-4423-a170-4315d35d31aa None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:13.057627 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 149/295] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:00:13.118500 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:13.122490 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b7ef3-64a2-4423-a170-4315d35d31aa None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:13.166626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d247d-df03-4a15-92e6-bec42e8dfacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:13.167288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d247d-df03-4a15-92e6-bec42e8dfacf None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:13.167566 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d247d-df03-4a15-92e6-bec42e8dfacf None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:13.167834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d247d-df03-4a15-92e6-bec42e8dfacf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:13.168577 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d247d-df03-4a15-92e6-bec42e8dfacf None admin] RBAC: Adding query filter params (role.id=e89162686187404da84651e0a3632b7b, scope.project.id=c57cd5c32d1447d3968cda9fc409de3c, user.id=affc11afd71f4f4687e2259f7362c0fe) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:13.168885 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d247d-df03-4a15-92e6-bec42e8dfacf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:13.170349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d247d-df03-4a15-92e6-bec42e8dfacf None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:13.177415 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 147/296] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:00:12 2022] GET /identity/v3/role_assignments?user.id=affc11afd71f4f4687e2259f7362c0fe&scope.project.id=c57cd5c32d1447d3968cda9fc409de3c&role.id=e89162686187404da84651e0a3632b7b => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:13.287320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2028f-4c94-4aca-bf70-a904d96f9f1c None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:13.294462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2028f-4c94-4aca-bf70-a904d96f9f1c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:13.295418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2028f-4c94-4aca-bf70-a904d96f9f1c None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:13.302613 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 150/297] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:13 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:13.305755 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bba309-e8f9-4d02-9d62-27b619e3700c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:13.306539 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bba309-e8f9-4d02-9d62-27b619e3700c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:13.306739 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bba309-e8f9-4d02-9d62-27b619e3700c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:13.347257 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b4bba309-e8f9-4d02-9d62-27b619e3700c None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:13.348463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b4bba309-e8f9-4d02-9d62-27b619e3700c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:13.390851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d2028f-4c94-4aca-bf70-a904d96f9f1c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:13.455753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 148/298] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:13.458479 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d62d7e-acb2-4713-84cf-3509c20c81d7 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:13.460102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d62d7e-acb2-4713-84cf-3509c20c81d7 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:13.460102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d62d7e-acb2-4713-84cf-3509c20c81d7 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:13.462952 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 149/299] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:13.469325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedccfa3-2a4a-47a3-92dc-ee6f9fee93a2 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:13.470495 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedccfa3-2a4a-47a3-92dc-ee6f9fee93a2 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:13.473007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedccfa3-2a4a-47a3-92dc-ee6f9fee93a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:13.517524 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-eedccfa3-2a4a-47a3-92dc-ee6f9fee93a2 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:13.518946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-eedccfa3-2a4a-47a3-92dc-ee6f9fee93a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:13.520157 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:13.615467 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d2028f-4c94-4aca-bf70-a904d96f9f1c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:13.628817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 150/300] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:13.678860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c9a6f-2711-49b2-b2eb-429c9e509b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:13.679817 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c9a6f-2711-49b2-b2eb-429c9e509b05 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:13.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c9a6f-2711-49b2-b2eb-429c9e509b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:13.680785 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c9a6f-2711-49b2-b2eb-429c9e509b05 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:13.701664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c9a6f-2711-49b2-b2eb-429c9e509b05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:13.713801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-fb1c9a6f-2711-49b2-b2eb-429c9e509b05 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:00:13.713801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:13.713801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:13.713801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:13.713801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:13.713801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:13.713801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:13.713801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:13.713801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:13.713801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:13.713801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:13.713801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:13.713801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:13.713801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:13.713801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:13.713801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:13.714784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:13.714784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:13.714784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:13.714784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:13.714784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:13.714784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:13.714784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:13.714784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:13.714784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:13.714784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:13.714784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:13.714784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:13.714784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:13.714784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:13.714784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:13.714784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:13.716363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:13.716363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:13.716363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:13.716363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:13.716363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:13.716363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:13.716363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:13.716363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:13.716363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:13.716363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:13.716363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:00:13.716363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:13.716363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 151/301] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:13.722303 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedccfa3-2a4a-47a3-92dc-ee6f9fee93a2 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:13.762128 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:13.877760 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedccfa3-2a4a-47a3-92dc-ee6f9fee93a2 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:13.934541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b2d7a-6659-4aad-99e7-54983006c4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:13.935560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b2d7a-6659-4aad-99e7-54983006c4ac None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:13.935976 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c9a6f-2711-49b2-b2eb-429c9e509b05 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:13.936586 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b2d7a-6659-4aad-99e7-54983006c4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:13.937077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b2d7a-6659-4aad-99e7-54983006c4ac None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:13.937961 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b2d7a-6659-4aad-99e7-54983006c4ac None admin] RBAC: Adding query filter params (name=reader) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:13.938479 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b2d7a-6659-4aad-99e7-54983006c4ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:13.940140 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b2d7a-6659-4aad-99e7-54983006c4ac None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:13.958503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 151/302] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 16:00:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:13.960418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b2d7a-6659-4aad-99e7-54983006c4ac None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:14.074871 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c9a6f-2711-49b2-b2eb-429c9e509b05 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:14.101937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:14.125557 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:14.134395 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699063e6-e99c-4684-9bd9-5ea535ad1256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:14.135062 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699063e6-e99c-4684-9bd9-5ea535ad1256 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:14.135372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699063e6-e99c-4684-9bd9-5ea535ad1256 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:14.135630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699063e6-e99c-4684-9bd9-5ea535ad1256 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:14.136869 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699063e6-e99c-4684-9bd9-5ea535ad1256 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:14.138343 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699063e6-e99c-4684-9bd9-5ea535ad1256 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:14.142722 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 152/303] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:14.142722 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699063e6-e99c-4684-9bd9-5ea535ad1256 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:14.166294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b2d7a-6659-4aad-99e7-54983006c4ac None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:14.220670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30959c4-ffdb-47f9-8c23-8e4efc927686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:14.221235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30959c4-ffdb-47f9-8c23-8e4efc927686 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:14.221511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30959c4-ffdb-47f9-8c23-8e4efc927686 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:14.221793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30959c4-ffdb-47f9-8c23-8e4efc927686 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:14.236502 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30959c4-ffdb-47f9-8c23-8e4efc927686 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:14.243514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-c30959c4-ffdb-47f9-8c23-8e4efc927686 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:14.243514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:14.243514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:14.243514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:14.243514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:14.243514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:14.243514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:14.243514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:14.243514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:14.243514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:14.243514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:14.243514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:14.243514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:14.243514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:14.243514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:14.243514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:14.244940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:14.244940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:14.244940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:14.244940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:14.244940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:14.244940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:14.244940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:14.244940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:14.244940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:14.244940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:14.244940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:14.244940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:14.244940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:14.244940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:14.244940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:14.244940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:14.246145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:14.246145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:14.246145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:14.246145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:14.246145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:14.246145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:14.246145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:14.246145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:14.246145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:14.246145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:14.246145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:14.246145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:14.246145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 152/304] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:14.246145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30959c4-ffdb-47f9-8c23-8e4efc927686 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:14.314611 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699063e6-e99c-4684-9bd9-5ea535ad1256 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:14.381400 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ce0d8-65cf-48b3-97c7-4b101a7d82c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:14.383029 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ce0d8-65cf-48b3-97c7-4b101a7d82c2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:14.383478 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ce0d8-65cf-48b3-97c7-4b101a7d82c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:14.383914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ce0d8-65cf-48b3-97c7-4b101a7d82c2 None admin] PATH_INFO: `/v3/users/10789a6f183a4b55abb7d50a3b5d54c3` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:14.408798 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ce0d8-65cf-48b3-97c7-4b101a7d82c2 None admin] RBAC: Authorizing `identity:get_user(user_id=10789a6f183a4b55abb7d50a3b5d54c3)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:14.410340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ce0d8-65cf-48b3-97c7-4b101a7d82c2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:14.421084 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 153/305] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:00:14 2022] GET /identity/v3/users/10789a6f183a4b55abb7d50a3b5d54c3 => generated 293 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:14.421084 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ce0d8-65cf-48b3-97c7-4b101a7d82c2 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:14.443032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30959c4-ffdb-47f9-8c23-8e4efc927686 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:14.485954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:14.511069 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b51de9-f3eb-4cf8-a01b-4ec3722b5042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:14.511702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b51de9-f3eb-4cf8-a01b-4ec3722b5042 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:14.512457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b51de9-f3eb-4cf8-a01b-4ec3722b5042 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:14.512729 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b51de9-f3eb-4cf8-a01b-4ec3722b5042 None admin] PATH_INFO: `/v3/groups` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:14.513708 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b51de9-f3eb-4cf8-a01b-4ec3722b5042 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:14.518720 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b51de9-f3eb-4cf8-a01b-4ec3722b5042 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:14.555450 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 153/306] 10.222.0.26 () {60 vars in 1134 bytes} [Tue Sep 6 16:00:14 2022] POST /identity/v3/groups => generated 229 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:00:14.556028 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b51de9-f3eb-4cf8-a01b-4ec3722b5042 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:14.561867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:14.630450 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ce0d8-65cf-48b3-97c7-4b101a7d82c2 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:14.717845 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3371b9-37a6-4809-812e-1af86692909e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:14.718473 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3371b9-37a6-4809-812e-1af86692909e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:14.718783 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3371b9-37a6-4809-812e-1af86692909e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:14.719087 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3371b9-37a6-4809-812e-1af86692909e None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:14.719729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3371b9-37a6-4809-812e-1af86692909e None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:14.720027 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3371b9-37a6-4809-812e-1af86692909e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:14.722688 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3371b9-37a6-4809-812e-1af86692909e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:14.737046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 154/307] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 16:00:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:14.743062 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3371b9-37a6-4809-812e-1af86692909e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:14.788105 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b51de9-f3eb-4cf8-a01b-4ec3722b5042 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:14.844243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77beaeed-7874-4b05-ad04-41a52daef2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:14.844861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77beaeed-7874-4b05-ad04-41a52daef2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:14.845468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77beaeed-7874-4b05-ad04-41a52daef2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:14.845772 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77beaeed-7874-4b05-ad04-41a52daef2b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:14.846830 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77beaeed-7874-4b05-ad04-41a52daef2b7 None admin] RBAC: Adding query filter params (role.id=11b32861fcd8469382d1a7047a95ea70, user.id=10789a6f183a4b55abb7d50a3b5d54c3, scope.system=all) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:14.847144 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77beaeed-7874-4b05-ad04-41a52daef2b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:14.849328 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77beaeed-7874-4b05-ad04-41a52daef2b7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:14.858475 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 154/308] 10.222.0.26 () {56 vars in 1308 bytes} [Tue Sep 6 16:00:14 2022] GET /identity/v3/role_assignments?user.id=10789a6f183a4b55abb7d50a3b5d54c3&scope.system=all&role.id=11b32861fcd8469382d1a7047a95ea70 => generated 226 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:14.876255 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3371b9-37a6-4809-812e-1af86692909e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:14.888935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:14.923706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70d46a-5103-481d-9ddd-e62489eb1313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:14.924370 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70d46a-5103-481d-9ddd-e62489eb1313 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:14.924900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70d46a-5103-481d-9ddd-e62489eb1313 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:14.929907 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70d46a-5103-481d-9ddd-e62489eb1313 None admin] PATH_INFO: `/v3/users/9f080f517c1a41e8b078196fae76a1dc` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:14.929907 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70d46a-5103-481d-9ddd-e62489eb1313 None admin] RBAC: Authorizing `identity:get_user(user_id=9f080f517c1a41e8b078196fae76a1dc)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:14.929907 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70d46a-5103-481d-9ddd-e62489eb1313 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:14.930601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 155/309] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:00:14 2022] GET /identity/v3/users/9f080f517c1a41e8b078196fae76a1dc => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:14.937190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77beaeed-7874-4b05-ad04-41a52daef2b7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:15.025322 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:15.099414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77beaeed-7874-4b05-ad04-41a52daef2b7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:15.162618 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63c11d-c115-4cb2-bf68-ee1094a6c52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:15.163540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63c11d-c115-4cb2-bf68-ee1094a6c52b None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:15.164117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63c11d-c115-4cb2-bf68-ee1094a6c52b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:15.164607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63c11d-c115-4cb2-bf68-ee1094a6c52b None admin] PATH_INFO: `/v3/system/users/9f080f517c1a41e8b078196fae76a1dc/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:15.167480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63c11d-c115-4cb2-bf68-ee1094a6c52b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9f080f517c1a41e8b078196fae76a1dc, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:15.170033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63c11d-c115-4cb2-bf68-ee1094a6c52b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:15.189715 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 155/310] 10.222.0.26 () {58 vars in 1334 bytes} [Tue Sep 6 16:00:14 2022] PUT /identity/v3/system/users/9f080f517c1a41e8b078196fae76a1dc/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:00:15.795042 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6651729-ac06-4f34-847d-aa8a7a47e28f None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:15.795571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6651729-ac06-4f34-847d-aa8a7a47e28f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:15.795571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6651729-ac06-4f34-847d-aa8a7a47e28f None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:15.797184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 156/311] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:15.802841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc0592-6452-4980-87d1-6798dade88bb None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:15.803932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc0592-6452-4980-87d1-6798dade88bb None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:15.803932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc0592-6452-4980-87d1-6798dade88bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:15.823810 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d9fc0592-6452-4980-87d1-6798dade88bb None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:15.825002 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d9fc0592-6452-4980-87d1-6798dade88bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:15.899526 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 156/312] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:16.185740 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6651729-ac06-4f34-847d-aa8a7a47e28f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:16.309454 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6651729-ac06-4f34-847d-aa8a7a47e28f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:16.354255 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab9be7-1ae6-4d5a-8dbf-a18ea1149719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:16.355308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab9be7-1ae6-4d5a-8dbf-a18ea1149719 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:16.355795 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab9be7-1ae6-4d5a-8dbf-a18ea1149719 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:16.356296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab9be7-1ae6-4d5a-8dbf-a18ea1149719 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:16.369440 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab9be7-1ae6-4d5a-8dbf-a18ea1149719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:16.375002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-bcab9be7-1ae6-4d5a-8dbf-a18ea1149719 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:00:16.375002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:16.375002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:16.375002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:16.375002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:16.375002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:16.375002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:16.375002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:16.375002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:16.375002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:16.375002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:16.375002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:16.375002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:16.375002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:16.375002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:16.375002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:16.375901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:16.375901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:16.375901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:16.375901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:16.375901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:16.375901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:16.375901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:16.375901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:16.375901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:16.375901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:16.375901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:16.375901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:16.375901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:16.375901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:16.375901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:16.375901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:16.376692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:16.376692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:16.376692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:16.376692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:16.376692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:16.376692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:16.376692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:16.376692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:16.376692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:16.376692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:16.376692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:00:16.376692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:16.379466 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 157/313] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:16.385802 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc0592-6452-4980-87d1-6798dade88bb None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:16.452565 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:16.495488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc0592-6452-4980-87d1-6798dade88bb None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:16.531595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53200d-32b8-4b16-970b-c58749195f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:16.532205 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53200d-32b8-4b16-970b-c58749195f42 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:16.532477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53200d-32b8-4b16-970b-c58749195f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:16.532742 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53200d-32b8-4b16-970b-c58749195f42 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:16.533409 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53200d-32b8-4b16-970b-c58749195f42 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:16.533761 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53200d-32b8-4b16-970b-c58749195f42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:16.535285 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53200d-32b8-4b16-970b-c58749195f42 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:16.544391 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 157/314] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 16:00:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:16.550195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab9be7-1ae6-4d5a-8dbf-a18ea1149719 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:16.594935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:16.666893 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab9be7-1ae6-4d5a-8dbf-a18ea1149719 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:16.705060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33451854-9237-431a-801e-90ef55a48ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:16.705627 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33451854-9237-431a-801e-90ef55a48ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:16.705909 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33451854-9237-431a-801e-90ef55a48ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:16.706183 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33451854-9237-431a-801e-90ef55a48ef3 None admin] PATH_INFO: `/v3/users/d8675a4b026c42049384ed8ed46cd28b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:16.707387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33451854-9237-431a-801e-90ef55a48ef3 None admin] RBAC: Authorizing `identity:get_user(user_id=d8675a4b026c42049384ed8ed46cd28b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:16.708839 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33451854-9237-431a-801e-90ef55a48ef3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:16.711441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 158/315] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:00:16 2022] GET /identity/v3/users/d8675a4b026c42049384ed8ed46cd28b => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:16.717170 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53200d-32b8-4b16-970b-c58749195f42 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:16.731186 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d5e27-98e6-473b-8132-4e5491587113 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:16.731494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d5e27-98e6-473b-8132-4e5491587113 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:16.731744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d5e27-98e6-473b-8132-4e5491587113 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:16.733419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 159/316] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:00:16.739014 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aeceae-65d6-4722-82ca-623ae4d8b70c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:16.739303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aeceae-65d6-4722-82ca-623ae4d8b70c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:16.739581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aeceae-65d6-4722-82ca-623ae4d8b70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:16.759273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-05aeceae-65d6-4722-82ca-623ae4d8b70c None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:16.760479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-05aeceae-65d6-4722-82ca-623ae4d8b70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:16.846351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53200d-32b8-4b16-970b-c58749195f42 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:16.879593 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:16.896150 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a4e98-9c63-43e3-ac22-a85db54d7610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:16.896779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a4e98-9c63-43e3-ac22-a85db54d7610 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:16.897144 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a4e98-9c63-43e3-ac22-a85db54d7610 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:16.897485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a4e98-9c63-43e3-ac22-a85db54d7610 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:16.903144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 160/317] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:00:16.984258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a4e98-9c63-43e3-ac22-a85db54d7610 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:16.985658 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a4e98-9c63-43e3-ac22-a85db54d7610 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:17.178213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 158/318] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:00:17.187550 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aeceae-65d6-4722-82ca-623ae4d8b70c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:17.235046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852bf65-054c-497b-b5e3-98f32485c33c None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:17.235436 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852bf65-054c-497b-b5e3-98f32485c33c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:17.235930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852bf65-054c-497b-b5e3-98f32485c33c None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:17.238853 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 159/319] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:00:17.248075 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b1f7b-59bc-40e9-b871-927b055a62c5 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:17.248696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b1f7b-59bc-40e9-b871-927b055a62c5 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:17.249267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b1f7b-59bc-40e9-b871-927b055a62c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:17.251424 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:17.283776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-339b1f7b-59bc-40e9-b871-927b055a62c5 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:17.285850 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-339b1f7b-59bc-40e9-b871-927b055a62c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:17.319005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:17.325065 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aeceae-65d6-4722-82ca-623ae4d8b70c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:17.363153 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e9416-a344-4629-bbfa-0aed20418924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:17.363784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e9416-a344-4629-bbfa-0aed20418924 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:17.364055 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e9416-a344-4629-bbfa-0aed20418924 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:17.364320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e9416-a344-4629-bbfa-0aed20418924 None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:17.365487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e9416-a344-4629-bbfa-0aed20418924 None admin] RBAC: Authorizing `identity:get_project(project_id=c57cd5c32d1447d3968cda9fc409de3c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:17.367906 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e9416-a344-4629-bbfa-0aed20418924 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:17.374585 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 161/320] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:17 2022] GET /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:17.374585 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e9416-a344-4629-bbfa-0aed20418924 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:17.415326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 160/321] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:17.415326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b1f7b-59bc-40e9-b871-927b055a62c5 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:17.578761 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e9416-a344-4629-bbfa-0aed20418924 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:17.581557 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:17.593929 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b1f7b-59bc-40e9-b871-927b055a62c5 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:17.648079 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e91a2-f0a2-4380-8b55-3d6561c47b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:17.649406 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e91a2-f0a2-4380-8b55-3d6561c47b51 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:17.649848 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e91a2-f0a2-4380-8b55-3d6561c47b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:17.650273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e91a2-f0a2-4380-8b55-3d6561c47b51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:17.652243 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e91a2-f0a2-4380-8b55-3d6561c47b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:17.655003 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e91a2-f0a2-4380-8b55-3d6561c47b51 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:17.658139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 162/322] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:17.658374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c564e2-7242-422d-bc8c-d8ca5546d263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:17.659085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c564e2-7242-422d-bc8c-d8ca5546d263 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:17.659523 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c564e2-7242-422d-bc8c-d8ca5546d263 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:17.659942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c564e2-7242-422d-bc8c-d8ca5546d263 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:17.660096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd94170-423a-4558-b64b-01393a24795d None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:17.660388 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd94170-423a-4558-b64b-01393a24795d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:17.660825 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd94170-423a-4558-b64b-01393a24795d None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:17.661310 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c564e2-7242-422d-bc8c-d8ca5546d263 None admin] RBAC: Adding query filter params (role.id=11b32861fcd8469382d1a7047a95ea70, scope.project.id=c57cd5c32d1447d3968cda9fc409de3c, user.id=d8675a4b026c42049384ed8ed46cd28b) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:17.661791 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c564e2-7242-422d-bc8c-d8ca5546d263 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:17.666869 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 163/323] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:17.666869 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd94170-423a-4558-b64b-01393a24795d None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:17.667228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c564e2-7242-422d-bc8c-d8ca5546d263 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:17.680436 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 161/324] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:00:17 2022] GET /identity/v3/role_assignments?user.id=d8675a4b026c42049384ed8ed46cd28b&scope.project.id=c57cd5c32d1447d3968cda9fc409de3c&role.id=11b32861fcd8469382d1a7047a95ea70 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:17.683505 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b93a8-802f-4830-9fa0-527b094fc890 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:17.683958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b93a8-802f-4830-9fa0-527b094fc890 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:17.684408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b93a8-802f-4830-9fa0-527b094fc890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:17.711351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-6c5b93a8-802f-4830-9fa0-527b094fc890 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:17.713194 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6c5b93a8-802f-4830-9fa0-527b094fc890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:17.732586 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:17.818401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 162/325] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:17.820999 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b93a8-802f-4830-9fa0-527b094fc890 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:17.831979 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd94170-423a-4558-b64b-01393a24795d None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:17.902974 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc069f-a32f-4697-9c49-c102e42a13bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:17.905295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc069f-a32f-4697-9c49-c102e42a13bc None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:17.907569 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc069f-a32f-4697-9c49-c102e42a13bc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:17.908348 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc069f-a32f-4697-9c49-c102e42a13bc None admin] PATH_INFO: `/v3/groups` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:17.910082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc069f-a32f-4697-9c49-c102e42a13bc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:17.915063 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc069f-a32f-4697-9c49-c102e42a13bc None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:17.952787 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 164/326] 10.222.0.26 () {60 vars in 1134 bytes} [Tue Sep 6 16:00:17 2022] POST /identity/v3/groups => generated 226 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:00:18.030219 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b93a8-802f-4830-9fa0-527b094fc890 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:18.093330 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110aca95-be41-4164-b255-a1b282acec6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:18.094129 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110aca95-be41-4164-b255-a1b282acec6e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:18.094593 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110aca95-be41-4164-b255-a1b282acec6e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:18.095112 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110aca95-be41-4164-b255-a1b282acec6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:18.111480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110aca95-be41-4164-b255-a1b282acec6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:18.119151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-110aca95-be41-4164-b255-a1b282acec6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:18.119151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:18.119151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:18.119151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:18.119151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:18.119151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:18.119151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:18.119151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:18.119151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:18.119151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:18.119151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:18.119151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:18.119151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:18.119151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:18.119151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:18.119151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:18.120231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:18.120231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:18.120231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:18.120231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:18.120231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:18.120231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:18.120231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:18.120231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:18.120231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:18.120231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:18.120231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:18.120231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:18.120231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:18.120231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:18.120231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:18.120231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:18.121712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:18.121712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:18.121712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:18.121712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:18.121712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:18.121712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:18.121712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:18.121712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:18.121712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:18.121712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:18.121712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:18.121712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:18.121712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 163/327] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:18.127935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc069f-a32f-4697-9c49-c102e42a13bc None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:18.156254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110aca95-be41-4164-b255-a1b282acec6e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:18.173738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:18.182241 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:18.324503 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc069f-a32f-4697-9c49-c102e42a13bc None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:18.364366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110aca95-be41-4164-b255-a1b282acec6e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:18.400485 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7e56b-6a9c-4f04-90a5-eddc9ccb7563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:18.401522 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7e56b-6a9c-4f04-90a5-eddc9ccb7563 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:18.401996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7e56b-6a9c-4f04-90a5-eddc9ccb7563 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:18.402454 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7e56b-6a9c-4f04-90a5-eddc9ccb7563 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:18.403760 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc7e56b-6a9c-4f04-90a5-eddc9ccb7563 None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:18.404251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc7e56b-6a9c-4f04-90a5-eddc9ccb7563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:18.406839 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc7e56b-6a9c-4f04-90a5-eddc9ccb7563 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:18.425785 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 165/328] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 16:00:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:18.430856 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fe17c-2a81-4d3f-a9f7-dced1671e762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:18.431822 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fe17c-2a81-4d3f-a9f7-dced1671e762 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:18.432306 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fe17c-2a81-4d3f-a9f7-dced1671e762 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:18.432794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fe17c-2a81-4d3f-a9f7-dced1671e762 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:18.434775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7e56b-6a9c-4f04-90a5-eddc9ccb7563 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:18.455101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4fe17c-2a81-4d3f-a9f7-dced1671e762 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:18.463160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-db4fe17c-2a81-4d3f-a9f7-dced1671e762 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:00:18.463160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:18.463160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:18.463160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:18.463160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:18.463160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:18.463160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:18.463160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:18.463160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:18.463160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:18.463160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:18.463160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:18.463160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:18.463160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:18.463160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:18.463160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:18.464845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:18.464845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:18.464845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:18.464845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:18.464845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:18.464845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:18.464845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:18.464845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:18.464845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:18.464845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:18.464845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:18.464845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:18.464845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:18.464845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:18.464845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:18.464845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:18.466355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:18.466355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:18.466355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:18.466355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:18.466355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:18.466355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:18.466355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:18.466355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:18.466355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:18.466355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:18.466355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:00:18.466355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:18.471000 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 164/329] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:00:18.471236 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fe17c-2a81-4d3f-a9f7-dced1671e762 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:18.586949 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:18.601027 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7e56b-6a9c-4f04-90a5-eddc9ccb7563 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:18.613390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:18.646446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1582a-93f5-4ecf-a46c-e186b90b5994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:18.647925 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1582a-93f5-4ecf-a46c-e186b90b5994 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:18.648374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1582a-93f5-4ecf-a46c-e186b90b5994 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:18.648813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1582a-93f5-4ecf-a46c-e186b90b5994 None admin] PATH_INFO: `/v3/users/9f080f517c1a41e8b078196fae76a1dc` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:18.650285 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1582a-93f5-4ecf-a46c-e186b90b5994 None admin] RBAC: Authorizing `identity:get_user(user_id=9f080f517c1a41e8b078196fae76a1dc)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:18.652078 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1582a-93f5-4ecf-a46c-e186b90b5994 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:18.658785 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 166/330] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:00:18 2022] GET /identity/v3/users/9f080f517c1a41e8b078196fae76a1dc => generated 293 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:18.660848 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1582a-93f5-4ecf-a46c-e186b90b5994 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:18.698749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fe17c-2a81-4d3f-a9f7-dced1671e762 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:18.788086 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e56578-1b35-4e27-8146-806332cf0c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:18.789685 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e56578-1b35-4e27-8146-806332cf0c73 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:18.790823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e56578-1b35-4e27-8146-806332cf0c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:18.791314 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e56578-1b35-4e27-8146-806332cf0c73 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:18.792911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e56578-1b35-4e27-8146-806332cf0c73 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:18.793425 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e56578-1b35-4e27-8146-806332cf0c73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:18.798418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e56578-1b35-4e27-8146-806332cf0c73 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:18.800118 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1582a-93f5-4ecf-a46c-e186b90b5994 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:18.828243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 165/331] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 16:00:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:18.832190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e56578-1b35-4e27-8146-806332cf0c73 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:18.856460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a2332-69a3-4259-8e9b-d4e527bcaea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:18.857391 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a2332-69a3-4259-8e9b-d4e527bcaea3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:18.857847 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a2332-69a3-4259-8e9b-d4e527bcaea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:18.858292 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a2332-69a3-4259-8e9b-d4e527bcaea3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:18.859278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a2332-69a3-4259-8e9b-d4e527bcaea3 None admin] RBAC: Adding query filter params (role.id=e89162686187404da84651e0a3632b7b, user.id=9f080f517c1a41e8b078196fae76a1dc, scope.system=all) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:18.859817 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a2332-69a3-4259-8e9b-d4e527bcaea3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:18.861457 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a2332-69a3-4259-8e9b-d4e527bcaea3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:18.883100 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 167/332] 10.222.0.26 () {56 vars in 1308 bytes} [Tue Sep 6 16:00:18 2022] GET /identity/v3/role_assignments?user.id=9f080f517c1a41e8b078196fae76a1dc&scope.system=all&role.id=e89162686187404da84651e0a3632b7b => generated 510 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:18.997432 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e56578-1b35-4e27-8146-806332cf0c73 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:19.036833 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:19.045140 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5fe6b-fcc3-4925-b58b-018f8788fa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:19.045798 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5fe6b-fcc3-4925-b58b-018f8788fa53 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:19.046074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5fe6b-fcc3-4925-b58b-018f8788fa53 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:19.046345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5fe6b-fcc3-4925-b58b-018f8788fa53 None admin] PATH_INFO: `/v3/users/10789a6f183a4b55abb7d50a3b5d54c3` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:19.047707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5fe6b-fcc3-4925-b58b-018f8788fa53 None admin] RBAC: Authorizing `identity:get_user(user_id=10789a6f183a4b55abb7d50a3b5d54c3)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:19.049186 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5fe6b-fcc3-4925-b58b-018f8788fa53 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:19.051235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 166/333] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:00:18 2022] GET /identity/v3/users/10789a6f183a4b55abb7d50a3b5d54c3 => generated 293 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:19.057917 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a2332-69a3-4259-8e9b-d4e527bcaea3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:19.091808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:19.202946 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a2332-69a3-4259-8e9b-d4e527bcaea3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:19.248001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2dcfd6-bd15-4c6d-87c4-9f17286a7242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:19.248664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2dcfd6-bd15-4c6d-87c4-9f17286a7242 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:19.248947 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2dcfd6-bd15-4c6d-87c4-9f17286a7242 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:19.249223 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2dcfd6-bd15-4c6d-87c4-9f17286a7242 None admin] PATH_INFO: `/v3/system/users/10789a6f183a4b55abb7d50a3b5d54c3/roles/11b32861fcd8469382d1a7047a95ea70` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:19.250840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2dcfd6-bd15-4c6d-87c4-9f17286a7242 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=10789a6f183a4b55abb7d50a3b5d54c3, role_id=11b32861fcd8469382d1a7047a95ea70)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:19.252244 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2dcfd6-bd15-4c6d-87c4-9f17286a7242 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:19.266774 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 168/334] 10.222.0.26 () {58 vars in 1334 bytes} [Tue Sep 6 16:00:19 2022] PUT /identity/v3/system/users/10789a6f183a4b55abb7d50a3b5d54c3/roles/11b32861fcd8469382d1a7047a95ea70 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:00:21.606206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45797653-5bf1-4728-909d-f44e96373b1a None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:21.606932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45797653-5bf1-4728-909d-f44e96373b1a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:21.607198 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45797653-5bf1-4728-909d-f44e96373b1a None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:21.609645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 167/335] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:21.630760 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bef10-505b-4d8a-8f90-b23c711620de None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:21.632060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bef10-505b-4d8a-8f90-b23c711620de None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:21.632658 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bef10-505b-4d8a-8f90-b23c711620de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:21.659781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-004bef10-505b-4d8a-8f90-b23c711620de None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:21.661331 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-004bef10-505b-4d8a-8f90-b23c711620de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:21.755857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 169/336] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:21.827102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea82c0f-6849-44b5-b62f-48597f394923 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:21.828014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea82c0f-6849-44b5-b62f-48597f394923 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:21.828743 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea82c0f-6849-44b5-b62f-48597f394923 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:21.831946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 168/337] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:21.844951 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55193898-dea2-49e3-adc6-5865bd72f577 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:21.845359 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55193898-dea2-49e3-adc6-5865bd72f577 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:21.845507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55193898-dea2-49e3-adc6-5865bd72f577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:21.878271 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-55193898-dea2-49e3-adc6-5865bd72f577 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:21.879417 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-55193898-dea2-49e3-adc6-5865bd72f577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:21.888896 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:21.899853 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e02d8-90ef-4e02-9fe5-16c72bf97c52 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:21.900754 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e02d8-90ef-4e02-9fe5-16c72bf97c52 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:21.901446 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e02d8-90ef-4e02-9fe5-16c72bf97c52 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:21.904778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 169/338] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:21.913244 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcf2db-4bec-491a-a8b2-c7807392f154 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:21.914365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcf2db-4bec-491a-a8b2-c7807392f154 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:21.915167 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcf2db-4bec-491a-a8b2-c7807392f154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:21.972461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-74bcf2db-4bec-491a-a8b2-c7807392f154 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:21.975618 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-74bcf2db-4bec-491a-a8b2-c7807392f154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:22.005401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 170/339] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:00:22.111980 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 170/340] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:22.265011 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55193898-dea2-49e3-adc6-5865bd72f577 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:22.396044 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55193898-dea2-49e3-adc6-5865bd72f577 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:22.447332 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a8b3e-415e-4350-8cba-0acb76ac7560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:22.448361 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a8b3e-415e-4350-8cba-0acb76ac7560 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:22.448865 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a8b3e-415e-4350-8cba-0acb76ac7560 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:22.449343 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a8b3e-415e-4350-8cba-0acb76ac7560 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:22.467648 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a8b3e-415e-4350-8cba-0acb76ac7560 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:22.473521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-055a8b3e-415e-4350-8cba-0acb76ac7560 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:22.473521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:22.473521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:22.473521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:22.473521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:22.473521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:22.473521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:22.473521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:22.473521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:22.473521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:22.473521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:22.473521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:22.473521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:22.473521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:22.473521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:22.473521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:22.474460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:22.474460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:22.474460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:22.474460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:22.474460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:22.474460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:22.474460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:22.474460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:22.474460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:22.474460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:22.474460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:22.474460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:22.474460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:22.474460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:22.474460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:22.474460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:22.475885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:22.475885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:22.475885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:22.475885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:22.475885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:22.475885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:22.475885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:22.475885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:22.475885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:22.475885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:22.475885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:22.475885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:22.478654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 171/341] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:22.484936 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcf2db-4bec-491a-a8b2-c7807392f154 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:22.622220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcf2db-4bec-491a-a8b2-c7807392f154 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:22.645432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a8b3e-415e-4350-8cba-0acb76ac7560 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:22.654718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:22.668390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:22.680403 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6945d16-d9c5-4f82-b69b-1cc861874ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:22.681041 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6945d16-d9c5-4f82-b69b-1cc861874ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:22.681325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6945d16-d9c5-4f82-b69b-1cc861874ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:22.681592 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6945d16-d9c5-4f82-b69b-1cc861874ed0 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:22.682276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6945d16-d9c5-4f82-b69b-1cc861874ed0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:22.682580 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6945d16-d9c5-4f82-b69b-1cc861874ed0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:22.684055 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6945d16-d9c5-4f82-b69b-1cc861874ed0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:22.701276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 171/342] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 16:00:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:22.706945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6945d16-d9c5-4f82-b69b-1cc861874ed0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:22.889415 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a8b3e-415e-4350-8cba-0acb76ac7560 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:22.914925 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6945d16-d9c5-4f82-b69b-1cc861874ed0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:22.954567 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad180e-81e5-4dc8-bca4-ce7f47db9805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:22.955254 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad180e-81e5-4dc8-bca4-ce7f47db9805 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:22.955542 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad180e-81e5-4dc8-bca4-ce7f47db9805 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:22.955817 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad180e-81e5-4dc8-bca4-ce7f47db9805 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:22.969313 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad180e-81e5-4dc8-bca4-ce7f47db9805 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:22.975679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-75ad180e-81e5-4dc8-bca4-ce7f47db9805 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:00:22.975679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:22.975679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:22.975679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:22.975679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:22.975679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:22.975679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:22.975679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:22.975679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:22.975679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:22.975679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:22.975679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:22.975679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:22.975679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:22.975679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:22.975679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:22.976609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:22.976609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:22.976609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:22.976609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:22.976609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:22.976609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:22.976609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:22.976609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:22.976609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:22.976609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:22.976609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:22.976609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:22.976609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:22.976609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:22.976609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:22.976609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:22.977425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:22.977425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:22.977425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:22.977425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:22.977425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:22.977425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:22.977425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:22.977425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:22.977425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:22.977425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:22.977425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:00:22.977425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:22.980218 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 172/343] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 16:00:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:22.981431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad180e-81e5-4dc8-bca4-ce7f47db9805 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:22.998145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d03fe-9189-4313-b4c3-8f162a8aa4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:22.999883 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d03fe-9189-4313-b4c3-8f162a8aa4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:23.000406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d03fe-9189-4313-b4c3-8f162a8aa4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:23.000907 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d03fe-9189-4313-b4c3-8f162a8aa4b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:23.081173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:23.096088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d03fe-9189-4313-b4c3-8f162a8aa4b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:23.097518 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d03fe-9189-4313-b4c3-8f162a8aa4b2 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:23.127078 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:23.187811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad180e-81e5-4dc8-bca4-ce7f47db9805 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:23.228676 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 172/344] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 16:00:23.230286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd65ae5-9972-4b69-91bb-233dd40587df None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:23.230286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd65ae5-9972-4b69-91bb-233dd40587df None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:23.230553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd65ae5-9972-4b69-91bb-233dd40587df None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:23.235021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 173/345] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:23.235021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd65ae5-9972-4b69-91bb-233dd40587df None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:23.267758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b825dfb-702c-4307-9a39-05eee9e88710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:23.269431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b825dfb-702c-4307-9a39-05eee9e88710 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:23.270303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b825dfb-702c-4307-9a39-05eee9e88710 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:23.271176 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b825dfb-702c-4307-9a39-05eee9e88710 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:23.295623 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b825dfb-702c-4307-9a39-05eee9e88710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:23.307656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-5b825dfb-702c-4307-9a39-05eee9e88710 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:00:23.307656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:23.307656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:23.307656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:23.307656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:23.307656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:23.307656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:23.307656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:23.307656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:23.307656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:23.307656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:23.307656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:23.307656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:23.307656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:23.307656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:23.307656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:23.309453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:23.309453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:23.309453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:23.309453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:23.309453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:23.309453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:23.309453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:23.309453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:23.309453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:23.309453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:23.309453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:23.309453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:23.309453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:23.309453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:23.309453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:23.309453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:23.310941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:23.310941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:23.310941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:23.310941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:23.310941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:23.310941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:23.310941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:23.310941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:23.310941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:23.310941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:23.310941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 16:00:23.310941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:23.315863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 173/346] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:23.318004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72d91d-9ece-465a-9843-ff8c1b687bdf None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:23.318467 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72d91d-9ece-465a-9843-ff8c1b687bdf None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:23.322017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72d91d-9ece-465a-9843-ff8c1b687bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:23.358245 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-cf72d91d-9ece-465a-9843-ff8c1b687bdf None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:23.358513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cf72d91d-9ece-465a-9843-ff8c1b687bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:23.379947 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:23.394772 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd65ae5-9972-4b69-91bb-233dd40587df None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:23.447217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a65ad-a091-444d-8124-7f753428176f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:23.447801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a65ad-a091-444d-8124-7f753428176f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:23.448088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a65ad-a091-444d-8124-7f753428176f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:23.448354 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a65ad-a091-444d-8124-7f753428176f None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:23.449034 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a65ad-a091-444d-8124-7f753428176f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:23.449342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a65ad-a091-444d-8124-7f753428176f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:23.453310 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a65ad-a091-444d-8124-7f753428176f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:23.461702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 174/347] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:00:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:23.462579 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a65ad-a091-444d-8124-7f753428176f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:23.479346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 174/348] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:23.481496 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3da57-654e-4766-83c5-9e9467ac11eb None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:23.481992 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3da57-654e-4766-83c5-9e9467ac11eb None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:23.482473 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3da57-654e-4766-83c5-9e9467ac11eb None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:23.486707 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 175/349] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:23.486707 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3da57-654e-4766-83c5-9e9467ac11eb None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:23.610121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:23.623660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a65ad-a091-444d-8124-7f753428176f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:23.677282 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a721e-2421-48c9-9fea-476ad56f95e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:23.679676 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a721e-2421-48c9-9fea-476ad56f95e8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:23.679965 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a721e-2421-48c9-9fea-476ad56f95e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:23.680239 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a721e-2421-48c9-9fea-476ad56f95e8 None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:23.682250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a721e-2421-48c9-9fea-476ad56f95e8 None admin] RBAC: Authorizing `identity:get_project(project_id=dda7941c8f444d41afc018b93fcd36fe)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:23.683227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a721e-2421-48c9-9fea-476ad56f95e8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:23.685193 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 175/350] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:23 2022] GET /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:23.686234 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b1279-99a1-4ab5-9efa-bf000357bbda None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:23.686654 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b1279-99a1-4ab5-9efa-bf000357bbda None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:23.686871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b1279-99a1-4ab5-9efa-bf000357bbda None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:23.688281 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 176/351] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:23.689326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b1279-99a1-4ab5-9efa-bf000357bbda None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:23.695729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3da57-654e-4766-83c5-9e9467ac11eb None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:23.717803 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:23.764572 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fc0d8-629f-452f-98d4-14ebc9ffad44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:23.766582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fc0d8-629f-452f-98d4-14ebc9ffad44 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:23.766582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fc0d8-629f-452f-98d4-14ebc9ffad44 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:23.766582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fc0d8-629f-452f-98d4-14ebc9ffad44 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:23.766582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fc0d8-629f-452f-98d4-14ebc9ffad44 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:23.768012 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fc0d8-629f-452f-98d4-14ebc9ffad44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:23.769223 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fc0d8-629f-452f-98d4-14ebc9ffad44 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:23.791755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 176/352] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 16:00:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:23.794355 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb506c20-e2ac-444f-a7a0-e368a30df658 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:23.796721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb506c20-e2ac-444f-a7a0-e368a30df658 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:23.796721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb506c20-e2ac-444f-a7a0-e368a30df658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:23.820392 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-cb506c20-e2ac-444f-a7a0-e368a30df658 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:23.821612 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cb506c20-e2ac-444f-a7a0-e368a30df658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:23.846375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b1279-99a1-4ab5-9efa-bf000357bbda None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:23.900715 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58356693-253e-453b-826f-719fa47187e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:23.902789 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58356693-253e-453b-826f-719fa47187e1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:23.903073 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58356693-253e-453b-826f-719fa47187e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:23.903346 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58356693-253e-453b-826f-719fa47187e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:23.908299 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 177/353] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:23.909265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb506c20-e2ac-444f-a7a0-e368a30df658 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:23.983608 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:24.022926 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58356693-253e-453b-826f-719fa47187e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:24.027385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58356693-253e-453b-826f-719fa47187e1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:24.066439 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:24.104019 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb506c20-e2ac-444f-a7a0-e368a30df658 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:24.181403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4cc53-0e65-44fb-be87-9581a621d198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:24.182453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4cc53-0e65-44fb-be87-9581a621d198 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:24.182968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4cc53-0e65-44fb-be87-9581a621d198 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:24.183559 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4cc53-0e65-44fb-be87-9581a621d198 None admin] PATH_INFO: `/v3/groups/4c446fe47fe64d38baacd39ac1b11320` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:24.195490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b4cc53-0e65-44fb-be87-9581a621d198 None admin] RBAC: Authorizing `identity:get_group(group_id=4c446fe47fe64d38baacd39ac1b11320)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:24.198145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b4cc53-0e65-44fb-be87-9581a621d198 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:24.201134 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 178/354] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 6 16:00:23 2022] GET /identity/v3/groups/4c446fe47fe64d38baacd39ac1b11320 => generated 226 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:24.203423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccde48-1fa5-4355-a7a3-78f5b603a4f9 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:24.203907 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccde48-1fa5-4355-a7a3-78f5b603a4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:24.204389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccde48-1fa5-4355-a7a3-78f5b603a4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:24.236948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d1ccde48-1fa5-4355-a7a3-78f5b603a4f9 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:24.238412 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d1ccde48-1fa5-4355-a7a3-78f5b603a4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:24.245074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 177/355] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:00:24.248553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58356693-253e-453b-826f-719fa47187e1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:24.362535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 179/356] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:24.363985 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ccde48-1fa5-4355-a7a3-78f5b603a4f9 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:24.426460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58356693-253e-453b-826f-719fa47187e1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:24.455527 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:24.471021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:24.496591 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d6aa6-f4f1-41b4-9a87-290fb7813163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:24.497772 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d6aa6-f4f1-41b4-9a87-290fb7813163 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:24.498261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d6aa6-f4f1-41b4-9a87-290fb7813163 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:24.498896 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d6aa6-f4f1-41b4-9a87-290fb7813163 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:24.512759 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d6aa6-f4f1-41b4-9a87-290fb7813163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:24.518503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-2b4d6aa6-f4f1-41b4-9a87-290fb7813163 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:24.518503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:24.518503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:24.518503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:24.518503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:24.518503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:24.518503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:24.518503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:24.518503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:24.518503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:24.518503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:24.518503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:24.518503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:24.518503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:24.518503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:24.518503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:24.519499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:24.519499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:24.519499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:24.519499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:24.519499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:24.519499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:24.519499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:24.519499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:24.519499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:24.519499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:24.519499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:24.519499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:24.519499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:24.519499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:24.519499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:24.519499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:24.520300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:24.520300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:24.520300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:24.520300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:24.520300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:24.520300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:24.520300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:24.520300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:24.520300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:24.520300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:24.520300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:24.520300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:24.522651 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 178/357] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:24.523732 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d6aa6-f4f1-41b4-9a87-290fb7813163 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:24.588698 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ccde48-1fa5-4355-a7a3-78f5b603a4f9 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:24.671959 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc0ab5-b2c5-4b33-9eae-2ac243c4a836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:24.672628 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc0ab5-b2c5-4b33-9eae-2ac243c4a836 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:24.672908 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc0ab5-b2c5-4b33-9eae-2ac243c4a836 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:24.673185 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc0ab5-b2c5-4b33-9eae-2ac243c4a836 None admin] PATH_INFO: `/v3/users/10789a6f183a4b55abb7d50a3b5d54c3` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:24.674438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc0ab5-b2c5-4b33-9eae-2ac243c4a836 None admin] RBAC: Authorizing `identity:get_user(user_id=10789a6f183a4b55abb7d50a3b5d54c3)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:24.676305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc0ab5-b2c5-4b33-9eae-2ac243c4a836 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:24.678748 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 180/358] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:00:24 2022] GET /identity/v3/users/10789a6f183a4b55abb7d50a3b5d54c3 => generated 293 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:24.679650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc0ab5-b2c5-4b33-9eae-2ac243c4a836 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:24.703497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d6aa6-f4f1-41b4-9a87-290fb7813163 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:24.779293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e949c0-464f-4952-9988-7dbe0295457d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:24.780629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e949c0-464f-4952-9988-7dbe0295457d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:24.781119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e949c0-464f-4952-9988-7dbe0295457d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:24.781560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e949c0-464f-4952-9988-7dbe0295457d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:24.784006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e949c0-464f-4952-9988-7dbe0295457d None admin] RBAC: Adding query filter params (group.id=4c446fe47fe64d38baacd39ac1b11320, role.id=e89162686187404da84651e0a3632b7b, scope.project.id=dda7941c8f444d41afc018b93fcd36fe) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:24.784006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e949c0-464f-4952-9988-7dbe0295457d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:24.786024 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e949c0-464f-4952-9988-7dbe0295457d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:24.798214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 179/359] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 6 16:00:24 2022] GET /identity/v3/role_assignments?group.id=4c446fe47fe64d38baacd39ac1b11320&scope.project.id=dda7941c8f444d41afc018b93fcd36fe&role.id=e89162686187404da84651e0a3632b7b => generated 260 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:24.800020 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e949c0-464f-4952-9988-7dbe0295457d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:24.852713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc0ab5-b2c5-4b33-9eae-2ac243c4a836 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:24.869747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:24.922698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:24.931876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d330e-7866-46b7-ba2a-36a984ab557c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:24.933297 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d330e-7866-46b7-ba2a-36a984ab557c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:24.934065 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d330e-7866-46b7-ba2a-36a984ab557c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:24.934603 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d330e-7866-46b7-ba2a-36a984ab557c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:24.954854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d330e-7866-46b7-ba2a-36a984ab557c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:24.964714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-548d330e-7866-46b7-ba2a-36a984ab557c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:00:24.964714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:24.964714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:24.964714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:24.964714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:24.964714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:24.964714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:24.964714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:24.964714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:24.964714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:24.964714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:24.964714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:24.964714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:24.964714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:24.964714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:24.964714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:24.966851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:24.966851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:24.966851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:24.966851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:24.966851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:24.966851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:24.966851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:24.966851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:24.966851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:24.966851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:24.966851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:24.966851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:24.966851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:24.966851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:24.966851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:24.966851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:24.968733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:24.968733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:24.968733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:24.968733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:24.968733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:24.968733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:24.968733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:24.968733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:24.968733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:24.968733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:24.968733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:00:24.968733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:24.970734 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 181/360] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 16:00:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:24.973238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d330e-7866-46b7-ba2a-36a984ab557c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:25.017708 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e949c0-464f-4952-9988-7dbe0295457d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:25.083255 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f9d61-1006-45f8-9b27-9e12b3e5512c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:25.085687 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f9d61-1006-45f8-9b27-9e12b3e5512c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:25.086158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f9d61-1006-45f8-9b27-9e12b3e5512c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:25.086335 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f9d61-1006-45f8-9b27-9e12b3e5512c None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:25.088253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f9d61-1006-45f8-9b27-9e12b3e5512c None admin] RBAC: Authorizing `identity:get_project(project_id=dda7941c8f444d41afc018b93fcd36fe)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:25.089727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f9d61-1006-45f8-9b27-9e12b3e5512c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:25.096181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 180/361] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:24 2022] GET /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:25.097174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f9d61-1006-45f8-9b27-9e12b3e5512c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:25.184294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d330e-7866-46b7-ba2a-36a984ab557c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:25.250112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f7dd3-b2d2-4ecd-be0b-43d0589f9d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:25.251037 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f7dd3-b2d2-4ecd-be0b-43d0589f9d4c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:25.251477 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f7dd3-b2d2-4ecd-be0b-43d0589f9d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:25.251710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f9d61-1006-45f8-9b27-9e12b3e5512c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:25.252043 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f7dd3-b2d2-4ecd-be0b-43d0589f9d4c None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:25.252868 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f7dd3-b2d2-4ecd-be0b-43d0589f9d4c None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:25.253320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f7dd3-b2d2-4ecd-be0b-43d0589f9d4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:25.257235 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f7dd3-b2d2-4ecd-be0b-43d0589f9d4c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:25.269663 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 182/362] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 16:00:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:25.271015 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f7dd3-b2d2-4ecd-be0b-43d0589f9d4c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:25.276515 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:25.329126 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6d762-23b1-4ec9-8d51-76e44728a138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:25.330087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6d762-23b1-4ec9-8d51-76e44728a138 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:25.330497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6d762-23b1-4ec9-8d51-76e44728a138 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:25.330998 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6d762-23b1-4ec9-8d51-76e44728a138 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:25.332133 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:25.332307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6d762-23b1-4ec9-8d51-76e44728a138 None admin] RBAC: Adding query filter params (role.id=11b32861fcd8469382d1a7047a95ea70, user.id=10789a6f183a4b55abb7d50a3b5d54c3, scope.system=all) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:25.332708 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6d762-23b1-4ec9-8d51-76e44728a138 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:25.336934 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6d762-23b1-4ec9-8d51-76e44728a138 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:25.355534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 181/363] 10.222.0.26 () {56 vars in 1308 bytes} [Tue Sep 6 16:00:25 2022] GET /identity/v3/role_assignments?user.id=10789a6f183a4b55abb7d50a3b5d54c3&scope.system=all&role.id=11b32861fcd8469382d1a7047a95ea70 => generated 510 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:25.356707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6d762-23b1-4ec9-8d51-76e44728a138 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:25.491089 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f7dd3-b2d2-4ecd-be0b-43d0589f9d4c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:25.538121 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6d762-23b1-4ec9-8d51-76e44728a138 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:25.543864 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56ffc3-15b7-4a1b-bde6-807be44cad03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:25.545080 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56ffc3-15b7-4a1b-bde6-807be44cad03 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:25.545646 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56ffc3-15b7-4a1b-bde6-807be44cad03 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:25.546116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56ffc3-15b7-4a1b-bde6-807be44cad03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:25.577778 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56ffc3-15b7-4a1b-bde6-807be44cad03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:25.584988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-ba56ffc3-15b7-4a1b-bde6-807be44cad03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:00:25.584988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:25.584988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:25.584988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:25.584988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:25.584988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:25.584988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:25.584988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:25.584988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:25.584988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:25.584988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:25.584988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:25.584988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:25.584988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:25.584988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:25.584988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:25.586006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:25.586006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:25.586006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:25.586006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:25.586006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:25.586006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:25.586006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:25.586006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:25.586006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:25.586006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:25.586006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:25.586006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:25.586006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:25.586006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:25.586006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:25.586006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:25.587298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:25.587298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:25.587298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:25.587298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:25.587298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:25.587298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:25.587298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:25.587298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:25.587298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:25.587298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:25.587298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:00:25.587298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:25.587298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 183/364] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:00:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:25.587298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56ffc3-15b7-4a1b-bde6-807be44cad03 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:25.613076 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3b91e-ef13-4f65-88bf-5576530c0789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:25.614362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3b91e-ef13-4f65-88bf-5576530c0789 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:25.614868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3b91e-ef13-4f65-88bf-5576530c0789 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:25.615849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3b91e-ef13-4f65-88bf-5576530c0789 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:25.617082 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3b91e-ef13-4f65-88bf-5576530c0789 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:25.617696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3b91e-ef13-4f65-88bf-5576530c0789 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:25.620237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3b91e-ef13-4f65-88bf-5576530c0789 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:25.635520 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 182/365] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:00:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:25.636998 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3b91e-ef13-4f65-88bf-5576530c0789 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:25.716740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:25.763516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:25.789254 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56ffc3-15b7-4a1b-bde6-807be44cad03 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:25.829636 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3b91e-ef13-4f65-88bf-5576530c0789 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:25.856731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b663d16-ddcd-4ece-886d-42646073166a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:25.857453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b663d16-ddcd-4ece-886d-42646073166a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:25.857737 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b663d16-ddcd-4ece-886d-42646073166a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:25.858062 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b663d16-ddcd-4ece-886d-42646073166a None admin] PATH_INFO: `/v3/groups/4c446fe47fe64d38baacd39ac1b11320` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:25.859266 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b663d16-ddcd-4ece-886d-42646073166a None admin] RBAC: Authorizing `identity:get_group(group_id=4c446fe47fe64d38baacd39ac1b11320)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:25.860732 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b663d16-ddcd-4ece-886d-42646073166a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:25.862739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 184/366] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 6 16:00:25 2022] GET /identity/v3/groups/4c446fe47fe64d38baacd39ac1b11320 => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:25.863722 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b663d16-ddcd-4ece-886d-42646073166a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:25.897847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88177b12-fcf7-47b7-a0b6-a42230edbd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:25.898492 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88177b12-fcf7-47b7-a0b6-a42230edbd29 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:25.898803 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88177b12-fcf7-47b7-a0b6-a42230edbd29 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:25.899309 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88177b12-fcf7-47b7-a0b6-a42230edbd29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:26.004136 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88177b12-fcf7-47b7-a0b6-a42230edbd29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:26.007146 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88177b12-fcf7-47b7-a0b6-a42230edbd29 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:26.042257 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b663d16-ddcd-4ece-886d-42646073166a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:26.063585 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:26.092099 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02863b-eaf2-44be-b910-a24416c99799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:26.093066 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02863b-eaf2-44be-b910-a24416c99799 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:26.093537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02863b-eaf2-44be-b910-a24416c99799 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:26.095916 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02863b-eaf2-44be-b910-a24416c99799 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:26.095916 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02863b-eaf2-44be-b910-a24416c99799 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:26.095916 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02863b-eaf2-44be-b910-a24416c99799 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:26.097896 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02863b-eaf2-44be-b910-a24416c99799 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:26.113738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 185/367] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:26.114308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02863b-eaf2-44be-b910-a24416c99799 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:26.143050 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:26.189247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 183/368] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:00:26.190847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88177b12-fcf7-47b7-a0b6-a42230edbd29 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:26.261859 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02863b-eaf2-44be-b910-a24416c99799 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:26.312555 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c7f48-2fde-4a9f-856a-9f78e340660e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:26.313183 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c7f48-2fde-4a9f-856a-9f78e340660e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:26.313472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c7f48-2fde-4a9f-856a-9f78e340660e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:26.313749 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c7f48-2fde-4a9f-856a-9f78e340660e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:26.351414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88177b12-fcf7-47b7-a0b6-a42230edbd29 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:26.389860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:26.396145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc99ce-5010-4fe1-97be-075835153ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:26.398200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc99ce-5010-4fe1-97be-075835153ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:26.399071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc99ce-5010-4fe1-97be-075835153ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:26.399394 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc99ce-5010-4fe1-97be-075835153ae0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:26.400085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc99ce-5010-4fe1-97be-075835153ae0 None admin] RBAC: Adding query filter params (group.id=4c446fe47fe64d38baacd39ac1b11320, role.id=bad9ff35005240e99bedef3fddb1dc77, scope.project.id=dda7941c8f444d41afc018b93fcd36fe) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:26.400397 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc99ce-5010-4fe1-97be-075835153ae0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:26.401880 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc99ce-5010-4fe1-97be-075835153ae0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:26.408170 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 184/369] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 6 16:00:26 2022] GET /identity/v3/role_assignments?group.id=4c446fe47fe64d38baacd39ac1b11320&scope.project.id=dda7941c8f444d41afc018b93fcd36fe&role.id=bad9ff35005240e99bedef3fddb1dc77 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:26.408170 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc99ce-5010-4fe1-97be-075835153ae0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:26.448064 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c7f48-2fde-4a9f-856a-9f78e340660e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:26.451486 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c7f48-2fde-4a9f-856a-9f78e340660e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:26.497795 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:26.632945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc99ce-5010-4fe1-97be-075835153ae0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:26.656639 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 186/370] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:00:26.656639 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c7f48-2fde-4a9f-856a-9f78e340660e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:26.696501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02638d48-c9f1-49a0-a85a-e7c61ffe3da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:26.697541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02638d48-c9f1-49a0-a85a-e7c61ffe3da5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:26.698054 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02638d48-c9f1-49a0-a85a-e7c61ffe3da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:26.698547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02638d48-c9f1-49a0-a85a-e7c61ffe3da5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:26.823638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:26.840030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02638d48-c9f1-49a0-a85a-e7c61ffe3da5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:26.840301 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c7f48-2fde-4a9f-856a-9f78e340660e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:26.843123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02638d48-c9f1-49a0-a85a-e7c61ffe3da5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:26.872526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:26.906551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be43739-380a-478b-970c-80b015bde7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:26.907301 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be43739-380a-478b-970c-80b015bde7da None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:26.907708 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be43739-380a-478b-970c-80b015bde7da None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:26.908189 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be43739-380a-478b-970c-80b015bde7da None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:26.910245 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be43739-380a-478b-970c-80b015bde7da None admin] RBAC: Authorizing `identity:get_project(project_id=c57cd5c32d1447d3968cda9fc409de3c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:26.913695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be43739-380a-478b-970c-80b015bde7da None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:26.917113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 187/371] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:26 2022] GET /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:26.918538 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be43739-380a-478b-970c-80b015bde7da None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:27.050453 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 185/372] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:00:27.051643 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02638d48-c9f1-49a0-a85a-e7c61ffe3da5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:27.101467 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be43739-380a-478b-970c-80b015bde7da None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:27.167114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251f7a3-aa1a-45b1-ad7e-2bd1d7520f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:27.168187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251f7a3-aa1a-45b1-ad7e-2bd1d7520f6d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:27.168651 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251f7a3-aa1a-45b1-ad7e-2bd1d7520f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:27.169147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251f7a3-aa1a-45b1-ad7e-2bd1d7520f6d None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:27.171317 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251f7a3-aa1a-45b1-ad7e-2bd1d7520f6d None admin] RBAC: Authorizing `identity:get_project(project_id=c57cd5c32d1447d3968cda9fc409de3c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:27.174101 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251f7a3-aa1a-45b1-ad7e-2bd1d7520f6d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:27.177648 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 188/373] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:26 2022] GET /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:27.183336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251f7a3-aa1a-45b1-ad7e-2bd1d7520f6d None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:27.196120 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:27.207936 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02638d48-c9f1-49a0-a85a-e7c61ffe3da5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:27.260752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0f371-0d11-4b91-b952-c6ae36ef8530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:27.261425 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0f371-0d11-4b91-b952-c6ae36ef8530 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:27.261705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0f371-0d11-4b91-b952-c6ae36ef8530 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:27.262127 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0f371-0d11-4b91-b952-c6ae36ef8530 None admin] PATH_INFO: `/v3/groups/4c446fe47fe64d38baacd39ac1b11320` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:27.263844 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0f371-0d11-4b91-b952-c6ae36ef8530 None admin] RBAC: Authorizing `identity:get_group(group_id=4c446fe47fe64d38baacd39ac1b11320)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:27.265420 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0f371-0d11-4b91-b952-c6ae36ef8530 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:27.275325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 186/374] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 6 16:00:27 2022] GET /identity/v3/groups/4c446fe47fe64d38baacd39ac1b11320 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:27.275325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca8576-79e5-4b37-bb7f-8c26a96b0e11 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:27.275325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca8576-79e5-4b37-bb7f-8c26a96b0e11 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:27.275325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca8576-79e5-4b37-bb7f-8c26a96b0e11 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:27.275325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 187/375] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:27.275325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca8576-79e5-4b37-bb7f-8c26a96b0e11 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:27.304504 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:27.397932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251f7a3-aa1a-45b1-ad7e-2bd1d7520f6d None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:27.422810 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca8576-79e5-4b37-bb7f-8c26a96b0e11 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:27.464104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104f49f-174e-49a9-ac2d-bd3c2562b0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:27.465162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104f49f-174e-49a9-ac2d-bd3c2562b0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:27.465665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104f49f-174e-49a9-ac2d-bd3c2562b0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:27.466160 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104f49f-174e-49a9-ac2d-bd3c2562b0f2 None admin] PATH_INFO: `/v3/projects/dcbe5a1252a54f0a914b3779a09b044d` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:27.468240 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104f49f-174e-49a9-ac2d-bd3c2562b0f2 None admin] RBAC: Authorizing `identity:get_project(project_id=dcbe5a1252a54f0a914b3779a09b044d)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:27.470980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104f49f-174e-49a9-ac2d-bd3c2562b0f2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:27.474141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 189/376] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:27 2022] GET /identity/v3/projects/dcbe5a1252a54f0a914b3779a09b044d => generated 344 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:27.476901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68b07a-d3bd-4926-a63f-b29067f592c6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:27.477395 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68b07a-d3bd-4926-a63f-b29067f592c6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:27.477876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68b07a-d3bd-4926-a63f-b29067f592c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:27.484055 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db03b98-2df4-4c88-90de-660b2008457b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:27.485069 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db03b98-2df4-4c88-90de-660b2008457b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:27.485537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db03b98-2df4-4c88-90de-660b2008457b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:27.486001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db03b98-2df4-4c88-90de-660b2008457b None admin] PATH_INFO: `/v3/groups/4c446fe47fe64d38baacd39ac1b11320` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:27.487834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db03b98-2df4-4c88-90de-660b2008457b None admin] RBAC: Authorizing `identity:get_group(group_id=4c446fe47fe64d38baacd39ac1b11320)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:27.490576 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db03b98-2df4-4c88-90de-660b2008457b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:27.498357 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 188/377] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 6 16:00:27 2022] GET /identity/v3/groups/4c446fe47fe64d38baacd39ac1b11320 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:27.498357 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db03b98-2df4-4c88-90de-660b2008457b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:27.509603 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-bc68b07a-d3bd-4926-a63f-b29067f592c6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:27.511747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bc68b07a-d3bd-4926-a63f-b29067f592c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:27.547516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:27.635389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 190/378] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:27.636554 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68b07a-d3bd-4926-a63f-b29067f592c6 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:27.693100 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db03b98-2df4-4c88-90de-660b2008457b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:27.726462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:27.788335 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c602d07-0a04-4430-9dc3-a2b2101f8684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:27.789864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c602d07-0a04-4430-9dc3-a2b2101f8684 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:27.790626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c602d07-0a04-4430-9dc3-a2b2101f8684 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:27.791426 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c602d07-0a04-4430-9dc3-a2b2101f8684 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:27.793094 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c602d07-0a04-4430-9dc3-a2b2101f8684 None admin] RBAC: Adding query filter params (group.id=4c446fe47fe64d38baacd39ac1b11320, role.id=e89162686187404da84651e0a3632b7b, scope.project.id=c57cd5c32d1447d3968cda9fc409de3c) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:27.793921 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c602d07-0a04-4430-9dc3-a2b2101f8684 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:27.797049 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c602d07-0a04-4430-9dc3-a2b2101f8684 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:27.809951 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 189/379] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 6 16:00:27 2022] GET /identity/v3/role_assignments?group.id=4c446fe47fe64d38baacd39ac1b11320&scope.project.id=c57cd5c32d1447d3968cda9fc409de3c&role.id=e89162686187404da84651e0a3632b7b => generated 260 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:27.812338 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c602d07-0a04-4430-9dc3-a2b2101f8684 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:27.846284 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68b07a-d3bd-4926-a63f-b29067f592c6 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:27.970126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe7ba3-8573-47d9-93e8-fb4535c73687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:27.971177 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe7ba3-8573-47d9-93e8-fb4535c73687 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:27.971628 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe7ba3-8573-47d9-93e8-fb4535c73687 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:27.972072 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe7ba3-8573-47d9-93e8-fb4535c73687 None admin] PATH_INFO: `/v3/groups/324b3694e04548d698dc3f058c0ccd69` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:27.982187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe7ba3-8573-47d9-93e8-fb4535c73687 None admin] RBAC: Authorizing `identity:get_group(group_id=324b3694e04548d698dc3f058c0ccd69)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:27.985050 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe7ba3-8573-47d9-93e8-fb4535c73687 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:27.987779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 191/380] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 6 16:00:27 2022] GET /identity/v3/groups/324b3694e04548d698dc3f058c0ccd69 => generated 229 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:27.989020 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe7ba3-8573-47d9-93e8-fb4535c73687 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:28.003986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:28.039708 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c602d07-0a04-4430-9dc3-a2b2101f8684 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:28.113031 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aabc83-2f86-4ee9-989d-cabdcadeab69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:28.114553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aabc83-2f86-4ee9-989d-cabdcadeab69 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:28.116254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aabc83-2f86-4ee9-989d-cabdcadeab69 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:28.117244 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aabc83-2f86-4ee9-989d-cabdcadeab69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:28.119219 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aabc83-2f86-4ee9-989d-cabdcadeab69 None admin] RBAC: Adding query filter params (group.id=4c446fe47fe64d38baacd39ac1b11320, role.id=bad9ff35005240e99bedef3fddb1dc77, scope.project.id=c57cd5c32d1447d3968cda9fc409de3c) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:28.121066 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aabc83-2f86-4ee9-989d-cabdcadeab69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:28.124527 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aabc83-2f86-4ee9-989d-cabdcadeab69 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:28.137138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 190/381] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 6 16:00:27 2022] GET /identity/v3/role_assignments?group.id=4c446fe47fe64d38baacd39ac1b11320&scope.project.id=c57cd5c32d1447d3968cda9fc409de3c&role.id=bad9ff35005240e99bedef3fddb1dc77 => generated 260 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:28.139833 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aabc83-2f86-4ee9-989d-cabdcadeab69 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:28.195911 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe7ba3-8573-47d9-93e8-fb4535c73687 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:28.231961 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:28.285402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5759ec5-321c-4ef7-842f-6521cd2e155d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:28.286088 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5759ec5-321c-4ef7-842f-6521cd2e155d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:28.286653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5759ec5-321c-4ef7-842f-6521cd2e155d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:28.287779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5759ec5-321c-4ef7-842f-6521cd2e155d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:28.312520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5759ec5-321c-4ef7-842f-6521cd2e155d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:28.325921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-f5759ec5-321c-4ef7-842f-6521cd2e155d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:28.325921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:28.325921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:28.325921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:28.325921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:28.325921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:28.325921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:28.325921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:28.325921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:28.325921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:28.325921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:28.325921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:28.325921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:28.325921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:28.325921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:28.325921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:28.327918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:28.327918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:28.327918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:28.327918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:28.327918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:28.327918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:28.327918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:28.327918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:28.327918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:28.327918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:28.327918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:28.327918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:28.327918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:28.327918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:28.327918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:28.327918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:28.329532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:28.329532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:28.329532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:28.329532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:28.329532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:28.329532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:28.329532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:28.329532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:28.329532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:28.329532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:28.329532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:28.329532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:28.329532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 192/382] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:28.339690 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5759ec5-321c-4ef7-842f-6521cd2e155d None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:28.393908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aabc83-2f86-4ee9-989d-cabdcadeab69 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:28.470966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c723e-07ec-43d9-8ae2-a37cc86cc388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:28.478160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c723e-07ec-43d9-8ae2-a37cc86cc388 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:28.479517 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c723e-07ec-43d9-8ae2-a37cc86cc388 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:28.482302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c723e-07ec-43d9-8ae2-a37cc86cc388 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:28.484194 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c723e-07ec-43d9-8ae2-a37cc86cc388 None admin] RBAC: Adding query filter params (group.id=324b3694e04548d698dc3f058c0ccd69, role.id=ba05677650e046a48d5f19440d48fbf2, scope.project.id=dcbe5a1252a54f0a914b3779a09b044d) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:28.485519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c723e-07ec-43d9-8ae2-a37cc86cc388 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:28.492478 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c723e-07ec-43d9-8ae2-a37cc86cc388 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:28.505124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 191/383] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 6 16:00:28 2022] GET /identity/v3/role_assignments?group.id=324b3694e04548d698dc3f058c0ccd69&scope.project.id=dcbe5a1252a54f0a914b3779a09b044d&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 260 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:28.531836 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5759ec5-321c-4ef7-842f-6521cd2e155d None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:28.583313 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606a07e-6dc0-4904-bb1d-15f21820e694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:28.585584 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606a07e-6dc0-4904-bb1d-15f21820e694 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:28.586395 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606a07e-6dc0-4904-bb1d-15f21820e694 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:28.587270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606a07e-6dc0-4904-bb1d-15f21820e694 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:28.588985 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606a07e-6dc0-4904-bb1d-15f21820e694 None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:28.589851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606a07e-6dc0-4904-bb1d-15f21820e694 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:28.592986 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606a07e-6dc0-4904-bb1d-15f21820e694 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:28.623635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 193/384] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 16:00:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:28.630360 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b8942-597c-4af2-8fba-da4f0fb9ff53 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:28.631116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b8942-597c-4af2-8fba-da4f0fb9ff53 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:28.631630 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b8942-597c-4af2-8fba-da4f0fb9ff53 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:28.634492 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 192/385] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:28.636121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606a07e-6dc0-4904-bb1d-15f21820e694 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:28.644394 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21264f4-545e-498d-b46c-10f877a5c631 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:28.644703 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21264f4-545e-498d-b46c-10f877a5c631 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:28.644952 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21264f4-545e-498d-b46c-10f877a5c631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:28.691918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-c21264f4-545e-498d-b46c-10f877a5c631 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:28.693213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c21264f4-545e-498d-b46c-10f877a5c631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:28.728761 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:28.728991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:28.833898 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606a07e-6dc0-4904-bb1d-15f21820e694 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:28.861125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 193/386] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:28.883945 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4567a9-00d7-41d1-91c6-39a2e5f797f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:28.884607 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4567a9-00d7-41d1-91c6-39a2e5f797f2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:28.884887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4567a9-00d7-41d1-91c6-39a2e5f797f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:28.885159 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4567a9-00d7-41d1-91c6-39a2e5f797f2 None admin] PATH_INFO: `/v3/groups/4c446fe47fe64d38baacd39ac1b11320` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:28.886340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4567a9-00d7-41d1-91c6-39a2e5f797f2 None admin] RBAC: Authorizing `identity:get_group(group_id=4c446fe47fe64d38baacd39ac1b11320)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:28.887826 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4567a9-00d7-41d1-91c6-39a2e5f797f2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:28.889664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 194/387] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 6 16:00:28 2022] GET /identity/v3/groups/4c446fe47fe64d38baacd39ac1b11320 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:28.895201 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21264f4-545e-498d-b46c-10f877a5c631 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:29.127716 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21264f4-545e-498d-b46c-10f877a5c631 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:29.207364 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498914ac-c6af-4d92-9b12-6df984d148e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:29.208462 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498914ac-c6af-4d92-9b12-6df984d148e4 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:29.208888 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498914ac-c6af-4d92-9b12-6df984d148e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:29.210802 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498914ac-c6af-4d92-9b12-6df984d148e4 None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:29.213146 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498914ac-c6af-4d92-9b12-6df984d148e4 None admin] RBAC: Authorizing `identity:get_project(project_id=dda7941c8f444d41afc018b93fcd36fe)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:29.217060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498914ac-c6af-4d92-9b12-6df984d148e4 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:29.220810 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 194/388] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:28 2022] GET /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe => generated 298 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:29.227143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4567a9-00d7-41d1-91c6-39a2e5f797f2 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:29.396233 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4567a9-00d7-41d1-91c6-39a2e5f797f2 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:29.409396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:29.423801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498914ac-c6af-4d92-9b12-6df984d148e4 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:29.449818 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa7fbe-5a45-4f56-92a5-0c2bca40f5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:29.450938 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa7fbe-5a45-4f56-92a5-0c2bca40f5af None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:29.451404 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa7fbe-5a45-4f56-92a5-0c2bca40f5af None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:29.451862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa7fbe-5a45-4f56-92a5-0c2bca40f5af None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe/groups/4c446fe47fe64d38baacd39ac1b11320/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:29.460199 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa7fbe-5a45-4f56-92a5-0c2bca40f5af None admin] RBAC: Authorizing `identity:create_grant(project_id=dda7941c8f444d41afc018b93fcd36fe, group_id=4c446fe47fe64d38baacd39ac1b11320, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:29.462971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa7fbe-5a45-4f56-92a5-0c2bca40f5af None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:29.479184 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:29.498452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 195/389] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Sep 6 16:00:29 2022] PUT /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe/groups/4c446fe47fe64d38baacd39ac1b11320/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:00:29.611615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498914ac-c6af-4d92-9b12-6df984d148e4 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:29.651299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110aa14e-0d4e-4dbe-a707-3362fa926da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:29.651875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110aa14e-0d4e-4dbe-a707-3362fa926da8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:29.652148 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110aa14e-0d4e-4dbe-a707-3362fa926da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:29.652414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110aa14e-0d4e-4dbe-a707-3362fa926da8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:29.664826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110aa14e-0d4e-4dbe-a707-3362fa926da8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:29.669918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-110aa14e-0d4e-4dbe-a707-3362fa926da8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:00:29.669918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:29.669918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:29.669918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:29.669918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:29.669918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:29.669918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:29.669918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:29.669918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:29.669918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:29.669918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:29.669918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:29.669918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:29.669918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:29.669918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:29.669918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:29.670875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:29.670875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:29.670875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:29.670875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:29.670875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:29.670875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:29.670875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:29.670875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:29.670875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:29.670875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:29.670875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:29.670875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:29.670875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:29.670875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:29.670875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:29.670875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:29.671653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:29.671653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:29.671653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:29.671653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:29.671653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:29.671653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:29.671653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:29.671653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:29.671653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:29.671653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:29.671653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:00:29.671653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:29.671653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 195/390] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 16:00:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:29.680506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa7fbe-5a45-4f56-92a5-0c2bca40f5af None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:29.887231 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa7fbe-5a45-4f56-92a5-0c2bca40f5af None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:29.931701 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b1d7c-9538-4b6b-bbcb-44b87aeb2a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:29.932641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b1d7c-9538-4b6b-bbcb-44b87aeb2a74 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:29.933102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b1d7c-9538-4b6b-bbcb-44b87aeb2a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:29.933560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b1d7c-9538-4b6b-bbcb-44b87aeb2a74 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:29.934424 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b1d7c-9538-4b6b-bbcb-44b87aeb2a74 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:29.934938 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b1d7c-9538-4b6b-bbcb-44b87aeb2a74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:29.936532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b1d7c-9538-4b6b-bbcb-44b87aeb2a74 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:29.947792 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 196/391] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:00:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:29.954523 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110aa14e-0d4e-4dbe-a707-3362fa926da8 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:30.069347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:30.106347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110aa14e-0d4e-4dbe-a707-3362fa926da8 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:30.165497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b078bf9-dfac-4d80-a7cc-f32d21a5c01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:30.166872 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b078bf9-dfac-4d80-a7cc-f32d21a5c01a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:30.167331 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b078bf9-dfac-4d80-a7cc-f32d21a5c01a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:30.167749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b078bf9-dfac-4d80-a7cc-f32d21a5c01a None admin] PATH_INFO: `/v3/groups/4c446fe47fe64d38baacd39ac1b11320` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:30.169110 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b078bf9-dfac-4d80-a7cc-f32d21a5c01a None admin] RBAC: Authorizing `identity:get_group(group_id=4c446fe47fe64d38baacd39ac1b11320)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:30.170914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b078bf9-dfac-4d80-a7cc-f32d21a5c01a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:30.180046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b1d7c-9538-4b6b-bbcb-44b87aeb2a74 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:30.181433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 196/392] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 6 16:00:29 2022] GET /identity/v3/groups/4c446fe47fe64d38baacd39ac1b11320 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:30.262033 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:30.424980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b1d7c-9538-4b6b-bbcb-44b87aeb2a74 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:30.491974 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceef490-00c7-4e50-b73e-72c9b21a1ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:30.493132 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceef490-00c7-4e50-b73e-72c9b21a1ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:30.493620 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceef490-00c7-4e50-b73e-72c9b21a1ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:30.494104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceef490-00c7-4e50-b73e-72c9b21a1ad8 None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:30.500072 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceef490-00c7-4e50-b73e-72c9b21a1ad8 None admin] RBAC: Authorizing `identity:get_project(project_id=dda7941c8f444d41afc018b93fcd36fe)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:30.502888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceef490-00c7-4e50-b73e-72c9b21a1ad8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:30.507165 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 197/393] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:30 2022] GET /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe => generated 298 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:30.513605 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b078bf9-dfac-4d80-a7cc-f32d21a5c01a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:30.669912 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b078bf9-dfac-4d80-a7cc-f32d21a5c01a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:30.711861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0934488-1ef3-40e6-a1ec-a575053f32dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:30.712549 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0934488-1ef3-40e6-a1ec-a575053f32dc None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:30.712831 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0934488-1ef3-40e6-a1ec-a575053f32dc None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:30.713177 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0934488-1ef3-40e6-a1ec-a575053f32dc None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe/groups/4c446fe47fe64d38baacd39ac1b11320/roles/bad9ff35005240e99bedef3fddb1dc77` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:30.715501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0934488-1ef3-40e6-a1ec-a575053f32dc None admin] RBAC: Authorizing `identity:create_grant(project_id=dda7941c8f444d41afc018b93fcd36fe, group_id=4c446fe47fe64d38baacd39ac1b11320, role_id=bad9ff35005240e99bedef3fddb1dc77)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:30.716970 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0934488-1ef3-40e6-a1ec-a575053f32dc None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:30.738832 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 197/394] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Sep 6 16:00:30 2022] PUT /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe/groups/4c446fe47fe64d38baacd39ac1b11320/roles/bad9ff35005240e99bedef3fddb1dc77 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:00:30.877470 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33602e-adcc-41a8-8a16-e144b96dae45 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:30.878119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33602e-adcc-41a8-8a16-e144b96dae45 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:30.878623 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33602e-adcc-41a8-8a16-e144b96dae45 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:30.880562 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 198/395] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:00:30.886708 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67009f22-3489-4542-891a-c7b86aa06e23 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:30.887338 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67009f22-3489-4542-891a-c7b86aa06e23 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:30.887818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67009f22-3489-4542-891a-c7b86aa06e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:30.917121 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-67009f22-3489-4542-891a-c7b86aa06e23 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:30.918779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-67009f22-3489-4542-891a-c7b86aa06e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:30.925713 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:31.015827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 198/396] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:31.444293 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33602e-adcc-41a8-8a16-e144b96dae45 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:31.475677 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff130665-b0fe-4c35-86b4-e22167d4862a None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:31.476715 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff130665-b0fe-4c35-86b4-e22167d4862a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:31.477241 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff130665-b0fe-4c35-86b4-e22167d4862a None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:31.479431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 199/397] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:31.485832 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401799e3-228f-4d93-b8f5-6d4bc741c7bd None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:31.486439 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401799e3-228f-4d93-b8f5-6d4bc741c7bd None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:31.487053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401799e3-228f-4d93-b8f5-6d4bc741c7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:31.509473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-401799e3-228f-4d93-b8f5-6d4bc741c7bd None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:31.517579 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-401799e3-228f-4d93-b8f5-6d4bc741c7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:31.635227 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:31.642009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33602e-adcc-41a8-8a16-e144b96dae45 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:31.684291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025876bc-a218-4881-8dc4-1da2b5fd74b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:31.684937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025876bc-a218-4881-8dc4-1da2b5fd74b3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:31.685212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025876bc-a218-4881-8dc4-1da2b5fd74b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:31.685483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025876bc-a218-4881-8dc4-1da2b5fd74b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:31.698470 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025876bc-a218-4881-8dc4-1da2b5fd74b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:31.703587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-025876bc-a218-4881-8dc4-1da2b5fd74b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:00:31.703587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:31.703587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:31.703587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:31.703587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:31.703587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:31.703587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:31.703587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:31.703587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:31.703587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:31.703587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:31.703587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:31.703587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:31.703587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:31.703587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:31.703587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:31.705161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:31.705161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:31.705161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:31.705161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:31.705161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:31.705161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:31.705161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:31.705161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:31.705161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:31.705161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:31.705161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:31.705161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:31.705161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:31.705161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:31.705161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:31.705161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:31.707273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:31.707273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:31.707273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:31.707273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:31.707273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:31.707273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:31.707273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:31.707273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:31.707273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:31.707273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:31.707273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:00:31.707273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:31.707273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 199/398] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 16:00:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:31.715081 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025876bc-a218-4881-8dc4-1da2b5fd74b3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:31.739332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 200/399] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:00:31.742205 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f206540-a2dc-45a0-9297-e6c85c556c99 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:31.742633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f206540-a2dc-45a0-9297-e6c85c556c99 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:31.743076 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f206540-a2dc-45a0-9297-e6c85c556c99 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:31.745823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 201/400] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:31.758980 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6f280-e544-4443-87bf-81f9a4ea1c9d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:31.759407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6f280-e544-4443-87bf-81f9a4ea1c9d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:31.759783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6f280-e544-4443-87bf-81f9a4ea1c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:31.819219 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-dea6f280-e544-4443-87bf-81f9a4ea1c9d None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:31.821315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-dea6f280-e544-4443-87bf-81f9a4ea1c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:31.997876 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:32.016120 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 202/401] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:00:32.050817 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025876bc-a218-4881-8dc4-1da2b5fd74b3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:32.169399 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3791ae-a007-45be-af99-a26fa08afbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:32.170921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3791ae-a007-45be-af99-a26fa08afbcf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:32.171684 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3791ae-a007-45be-af99-a26fa08afbcf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:32.172452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3791ae-a007-45be-af99-a26fa08afbcf None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:32.173926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3791ae-a007-45be-af99-a26fa08afbcf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:32.179336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3791ae-a007-45be-af99-a26fa08afbcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:32.182434 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3791ae-a007-45be-af99-a26fa08afbcf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:32.220741 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 200/402] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:00:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 506 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:32.234419 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6f280-e544-4443-87bf-81f9a4ea1c9d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:32.393117 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3791ae-a007-45be-af99-a26fa08afbcf None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:32.473295 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6f280-e544-4443-87bf-81f9a4ea1c9d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:32.500807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:32.578904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e03da-2986-4697-b341-5ef6e24ee837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:32.579688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e03da-2986-4697-b341-5ef6e24ee837 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:32.579864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e03da-2986-4697-b341-5ef6e24ee837 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:32.580185 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e03da-2986-4697-b341-5ef6e24ee837 None admin] PATH_INFO: `/v3/groups/4c446fe47fe64d38baacd39ac1b11320` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:32.582690 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e03da-2986-4697-b341-5ef6e24ee837 None admin] RBAC: Authorizing `identity:get_group(group_id=4c446fe47fe64d38baacd39ac1b11320)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:32.582690 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e03da-2986-4697-b341-5ef6e24ee837 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:32.585208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 203/403] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 6 16:00:32 2022] GET /identity/v3/groups/4c446fe47fe64d38baacd39ac1b11320 => generated 226 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:32.594592 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e03da-2986-4697-b341-5ef6e24ee837 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:32.705268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:32.757084 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e03da-2986-4697-b341-5ef6e24ee837 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:32.792618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3791ae-a007-45be-af99-a26fa08afbcf None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:32.811640 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39863869-05c4-4d82-b2d6-a388b781a1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:32.812562 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39863869-05c4-4d82-b2d6-a388b781a1e1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:32.813261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39863869-05c4-4d82-b2d6-a388b781a1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:32.814871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39863869-05c4-4d82-b2d6-a388b781a1e1 None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:32.815782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39863869-05c4-4d82-b2d6-a388b781a1e1 None admin] RBAC: Authorizing `identity:get_project(project_id=c57cd5c32d1447d3968cda9fc409de3c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:32.820036 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39863869-05c4-4d82-b2d6-a388b781a1e1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:32.823435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 204/404] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:32 2022] GET /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:32.825496 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff27e1-2f8a-4294-b08d-c03dea1b282c None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:32.825991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff27e1-2f8a-4294-b08d-c03dea1b282c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:32.826476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff27e1-2f8a-4294-b08d-c03dea1b282c None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:32.829143 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 205/405] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:32.830451 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff27e1-2f8a-4294-b08d-c03dea1b282c None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:32.847751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27892bec-d447-4ce7-9ad7-9383ab0cbf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:32.848383 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27892bec-d447-4ce7-9ad7-9383ab0cbf75 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:32.848660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27892bec-d447-4ce7-9ad7-9383ab0cbf75 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:32.848937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27892bec-d447-4ce7-9ad7-9383ab0cbf75 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:32.861510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27892bec-d447-4ce7-9ad7-9383ab0cbf75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:32.867141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-27892bec-d447-4ce7-9ad7-9383ab0cbf75 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:32.867141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:32.867141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:32.867141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:32.867141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:32.867141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:32.867141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:32.867141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:32.867141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:32.867141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:32.867141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:32.867141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:32.867141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:32.867141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:32.867141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:32.867141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:32.868074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:32.868074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:32.868074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:32.868074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:32.868074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:32.868074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:32.868074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:32.868074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:32.868074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:32.868074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:32.868074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:32.868074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:32.868074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:32.868074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:32.868074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:32.868074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:32.868877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:32.868877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:32.868877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:32.868877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:32.868877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:32.868877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:32.868877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:32.868877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:32.868877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:32.868877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:32.868877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:32.868877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:32.873139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 201/406] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 481 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:32.874049 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27892bec-d447-4ce7-9ad7-9383ab0cbf75 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:32.962928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:32.996118 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27892bec-d447-4ce7-9ad7-9383ab0cbf75 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:33.006561 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff27e1-2f8a-4294-b08d-c03dea1b282c None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:33.047287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc1378-b032-4e20-a2e3-dc4b5a75188e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:33.048403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc1378-b032-4e20-a2e3-dc4b5a75188e None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:33.048900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc1378-b032-4e20-a2e3-dc4b5a75188e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:33.049389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc1378-b032-4e20-a2e3-dc4b5a75188e None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c/groups/4c446fe47fe64d38baacd39ac1b11320/roles/bad9ff35005240e99bedef3fddb1dc77` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:33.054008 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc1378-b032-4e20-a2e3-dc4b5a75188e None admin] RBAC: Authorizing `identity:create_grant(project_id=c57cd5c32d1447d3968cda9fc409de3c, group_id=4c446fe47fe64d38baacd39ac1b11320, role_id=bad9ff35005240e99bedef3fddb1dc77)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:33.055777 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a9cb1-41a5-4737-bc0d-4368a774134c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:33.056422 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a9cb1-41a5-4737-bc0d-4368a774134c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:33.056702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a9cb1-41a5-4737-bc0d-4368a774134c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:33.056887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc1378-b032-4e20-a2e3-dc4b5a75188e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:33.057034 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a9cb1-41a5-4737-bc0d-4368a774134c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:33.060638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:33.071373 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a9cb1-41a5-4737-bc0d-4368a774134c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:33.076660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-037a9cb1-41a5-4737-bc0d-4368a774134c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:00:33.076660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:33.076660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:33.076660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:33.076660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:33.076660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:33.076660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:33.076660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:33.076660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:33.076660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:33.076660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:33.076660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:33.076660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:33.076660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:33.076660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:33.076660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:33.077624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:33.077624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:33.077624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:33.077624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:33.077624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:33.077624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:33.077624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:33.077624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:33.077624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:33.077624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:33.077624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:33.077624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:33.077624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:33.077624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:33.077624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:33.077624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:33.079006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:33.079006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:33.079006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:33.079006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:33.079006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:33.079006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:33.079006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:33.079006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:33.079006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:33.079006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:33.079006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:00:33.079006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:33.079006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 206/407] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:00:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:33.080084 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1f896-7e9d-4a87-8cbe-128d709b8756 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:33.080370 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1f896-7e9d-4a87-8cbe-128d709b8756 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:33.080642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1f896-7e9d-4a87-8cbe-128d709b8756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:33.099906 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 202/408] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Sep 6 16:00:32 2022] PUT /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c/groups/4c446fe47fe64d38baacd39ac1b11320/roles/bad9ff35005240e99bedef3fddb1dc77 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:00:33.099906 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc1378-b032-4e20-a2e3-dc4b5a75188e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:33.105962 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b6e1f896-7e9d-4a87-8cbe-128d709b8756 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:33.108343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b6e1f896-7e9d-4a87-8cbe-128d709b8756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:33.211385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 207/409] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:33.212312 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1f896-7e9d-4a87-8cbe-128d709b8756 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:33.289683 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc1378-b032-4e20-a2e3-dc4b5a75188e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:33.358257 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5cf70-a292-4043-8ce6-4f1062461360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:33.359403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5cf70-a292-4043-8ce6-4f1062461360 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:33.359885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5cf70-a292-4043-8ce6-4f1062461360 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:33.360515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5cf70-a292-4043-8ce6-4f1062461360 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:33.361389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5cf70-a292-4043-8ce6-4f1062461360 None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:33.361890 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5cf70-a292-4043-8ce6-4f1062461360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:33.363493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5cf70-a292-4043-8ce6-4f1062461360 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:33.369643 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:33.374711 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 203/410] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 16:00:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:33.382574 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5cf70-a292-4043-8ce6-4f1062461360 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:33.415141 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1f896-7e9d-4a87-8cbe-128d709b8756 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:33.469692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:33.487701 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bf21f-7e54-4a3a-9561-2593ea7dbbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:33.488896 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bf21f-7e54-4a3a-9561-2593ea7dbbc9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:33.489383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bf21f-7e54-4a3a-9561-2593ea7dbbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:33.489835 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bf21f-7e54-4a3a-9561-2593ea7dbbc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:33.491149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14bf21f-7e54-4a3a-9561-2593ea7dbbc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:33.491675 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14bf21f-7e54-4a3a-9561-2593ea7dbbc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:33.494470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14bf21f-7e54-4a3a-9561-2593ea7dbbc9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:33.504642 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5cf70-a292-4043-8ce6-4f1062461360 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:33.513240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 208/411] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:33.524506 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bf21f-7e54-4a3a-9561-2593ea7dbbc9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:33.546245 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb18da5-3fa7-4fe7-8e92-934fe9991b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:33.547338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb18da5-3fa7-4fe7-8e92-934fe9991b4f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:33.547833 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb18da5-3fa7-4fe7-8e92-934fe9991b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:33.548298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb18da5-3fa7-4fe7-8e92-934fe9991b4f None admin] PATH_INFO: `/v3/groups/4c446fe47fe64d38baacd39ac1b11320` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:33.549742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb18da5-3fa7-4fe7-8e92-934fe9991b4f None admin] RBAC: Authorizing `identity:get_group(group_id=4c446fe47fe64d38baacd39ac1b11320)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:33.551809 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb18da5-3fa7-4fe7-8e92-934fe9991b4f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:33.558786 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 204/412] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 6 16:00:33 2022] GET /identity/v3/groups/4c446fe47fe64d38baacd39ac1b11320 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:33.560877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb18da5-3fa7-4fe7-8e92-934fe9991b4f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:33.711758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:33.766825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bf21f-7e54-4a3a-9561-2593ea7dbbc9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:33.779287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb18da5-3fa7-4fe7-8e92-934fe9991b4f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:33.828806 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30feada6-5bf0-47a1-93e6-90a7fd139c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:33.829873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30feada6-5bf0-47a1-93e6-90a7fd139c41 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:33.830351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30feada6-5bf0-47a1-93e6-90a7fd139c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:33.830891 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30feada6-5bf0-47a1-93e6-90a7fd139c41 None admin] PATH_INFO: `/v3/groups/324b3694e04548d698dc3f058c0ccd69` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:33.832824 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30feada6-5bf0-47a1-93e6-90a7fd139c41 None admin] RBAC: Authorizing `identity:get_group(group_id=324b3694e04548d698dc3f058c0ccd69)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:33.836147 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30feada6-5bf0-47a1-93e6-90a7fd139c41 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:33.843642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 209/413] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 6 16:00:33 2022] GET /identity/v3/groups/324b3694e04548d698dc3f058c0ccd69 => generated 229 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:33.843642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ff8e3-517b-4c8e-9666-a38ffce87358 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:33.843642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ff8e3-517b-4c8e-9666-a38ffce87358 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:33.843642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ff8e3-517b-4c8e-9666-a38ffce87358 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:33.845959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 210/414] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:33.846209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ff8e3-517b-4c8e-9666-a38ffce87358 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:33.848985 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701dfad-0daa-4539-a07b-3c02f0126178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:33.850071 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701dfad-0daa-4539-a07b-3c02f0126178 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:33.850561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701dfad-0daa-4539-a07b-3c02f0126178 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:33.851384 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701dfad-0daa-4539-a07b-3c02f0126178 None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:33.854841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701dfad-0daa-4539-a07b-3c02f0126178 None admin] RBAC: Authorizing `identity:get_project(project_id=c57cd5c32d1447d3968cda9fc409de3c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:33.857597 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701dfad-0daa-4539-a07b-3c02f0126178 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:33.863756 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 205/415] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:33 2022] GET /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:33.863756 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701dfad-0daa-4539-a07b-3c02f0126178 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:33.948955 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:34.015809 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ff8e3-517b-4c8e-9666-a38ffce87358 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:34.033254 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701dfad-0daa-4539-a07b-3c02f0126178 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:34.080454 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038bf72-10c0-4f18-bf09-a4519e07cfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:34.081377 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038bf72-10c0-4f18-bf09-a4519e07cfb5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:34.081813 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038bf72-10c0-4f18-bf09-a4519e07cfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:34.082247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038bf72-10c0-4f18-bf09-a4519e07cfb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:34.097684 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d176a2-93ce-4dae-a04c-9ac546315f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:34.099069 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d176a2-93ce-4dae-a04c-9ac546315f9a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:34.099191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038bf72-10c0-4f18-bf09-a4519e07cfb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:34.099808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d176a2-93ce-4dae-a04c-9ac546315f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:34.100539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d176a2-93ce-4dae-a04c-9ac546315f9a None admin] PATH_INFO: `/v3/projects/dcbe5a1252a54f0a914b3779a09b044d` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:34.102338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d176a2-93ce-4dae-a04c-9ac546315f9a None admin] RBAC: Authorizing `identity:get_project(project_id=dcbe5a1252a54f0a914b3779a09b044d)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:34.104273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d176a2-93ce-4dae-a04c-9ac546315f9a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:34.104413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-0038bf72-10c0-4f18-bf09-a4519e07cfb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:34.104413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:34.104413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:34.104413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:34.104413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:34.104413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:34.104413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:34.104413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:34.104413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:34.104413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:34.104413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:34.104413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:34.104413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:34.104413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:34.104413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:34.104413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:34.105291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:34.105291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:34.105291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:34.105291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:34.105291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:34.105291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:34.105291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:34.105291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:34.105291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:34.105291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:34.105291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:34.105291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:34.105291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:34.105291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:34.105291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:34.105291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:34.106629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:34.106629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:34.106629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:34.106629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:34.106629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:34.106629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:34.106629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:34.106629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:34.106629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:34.106629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:34.106629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:34.106629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:34.106629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 211/416] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:34.107938 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0da04-e012-4343-a894-24e766006719 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:34.107938 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0da04-e012-4343-a894-24e766006719 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:34.107938 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0da04-e012-4343-a894-24e766006719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:34.113516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 206/417] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:33 2022] GET /identity/v3/projects/dcbe5a1252a54f0a914b3779a09b044d => generated 344 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:34.117472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d176a2-93ce-4dae-a04c-9ac546315f9a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:34.125394 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:34.128903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-08b0da04-e012-4343-a894-24e766006719 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:34.130149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-08b0da04-e012-4343-a894-24e766006719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:34.220048 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 212/418] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:34.230949 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0da04-e012-4343-a894-24e766006719 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:34.287799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d176a2-93ce-4dae-a04c-9ac546315f9a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:34.291544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:34.353352 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0da04-e012-4343-a894-24e766006719 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:34.362426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c402ee-7752-4712-b9cf-ee359f69c901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:34.363727 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c402ee-7752-4712-b9cf-ee359f69c901 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:34.364347 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c402ee-7752-4712-b9cf-ee359f69c901 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:34.364844 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c402ee-7752-4712-b9cf-ee359f69c901 None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c/groups/4c446fe47fe64d38baacd39ac1b11320/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:34.368498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c402ee-7752-4712-b9cf-ee359f69c901 None admin] RBAC: Authorizing `identity:create_grant(project_id=c57cd5c32d1447d3968cda9fc409de3c, group_id=4c446fe47fe64d38baacd39ac1b11320, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:34.371235 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c402ee-7752-4712-b9cf-ee359f69c901 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:34.391213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dece1-5dc5-442f-a9e0-179bbdb5c21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:34.392057 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dece1-5dc5-442f-a9e0-179bbdb5c21e None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:34.392336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dece1-5dc5-442f-a9e0-179bbdb5c21e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:34.392606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dece1-5dc5-442f-a9e0-179bbdb5c21e None admin] PATH_INFO: `/v3/projects/dcbe5a1252a54f0a914b3779a09b044d/groups/324b3694e04548d698dc3f058c0ccd69/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:34.394959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6dece1-5dc5-442f-a9e0-179bbdb5c21e None admin] RBAC: Authorizing `identity:create_grant(project_id=dcbe5a1252a54f0a914b3779a09b044d, group_id=324b3694e04548d698dc3f058c0ccd69, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:34.397199 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6dece1-5dc5-442f-a9e0-179bbdb5c21e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:34.400230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 207/419] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Sep 6 16:00:34 2022] PUT /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c/groups/4c446fe47fe64d38baacd39ac1b11320/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:00:34.402559 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c402ee-7752-4712-b9cf-ee359f69c901 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:34.429060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 213/420] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Sep 6 16:00:34 2022] PUT /identity/v3/projects/dcbe5a1252a54f0a914b3779a09b044d/groups/324b3694e04548d698dc3f058c0ccd69/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:00:34.536991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:34.577049 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dece1-5dc5-442f-a9e0-179bbdb5c21e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:34.650533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c402ee-7752-4712-b9cf-ee359f69c901 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:34.732194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfb056-fe20-4514-812e-62fcb133a460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:34.733228 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfb056-fe20-4514-812e-62fcb133a460 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:34.733788 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfb056-fe20-4514-812e-62fcb133a460 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:34.734267 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfb056-fe20-4514-812e-62fcb133a460 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:34.735540 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfb056-fe20-4514-812e-62fcb133a460 None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:34.736048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfb056-fe20-4514-812e-62fcb133a460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:34.738535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfb056-fe20-4514-812e-62fcb133a460 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:34.762707 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 208/421] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 16:00:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:34.767726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfb056-fe20-4514-812e-62fcb133a460 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:34.835200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dece1-5dc5-442f-a9e0-179bbdb5c21e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:34.863776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:34.962018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56002d50-64d4-4662-9b86-7f33edc3fec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:34.962566 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56002d50-64d4-4662-9b86-7f33edc3fec4 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:34.962875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56002d50-64d4-4662-9b86-7f33edc3fec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:34.963150 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56002d50-64d4-4662-9b86-7f33edc3fec4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:34.986077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56002d50-64d4-4662-9b86-7f33edc3fec4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:34.991898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-56002d50-64d4-4662-9b86-7f33edc3fec4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:00:34.991898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:34.991898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:34.991898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:34.991898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:34.991898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:34.991898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:34.991898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:34.991898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:34.991898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:34.991898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:34.991898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:34.991898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:34.991898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:34.991898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:34.991898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:34.993588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:34.993588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:34.993588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:34.993588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:34.993588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:34.993588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:34.993588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:34.993588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:34.993588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:34.993588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:34.993588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:34.993588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:34.993588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:34.993588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:34.993588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:34.993588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:34.995960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:34.995960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:34.995960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:34.995960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:34.995960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:34.995960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:34.995960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:34.995960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:34.995960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:34.995960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:34.995960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:00:34.995960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:34.995960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 214/422] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 16:00:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:00:35.002709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56002d50-64d4-4662-9b86-7f33edc3fec4 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:35.064389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfb056-fe20-4514-812e-62fcb133a460 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:35.127283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:35.142004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367d0c0-7f61-4da6-a376-c03cbbe9b5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:35.143582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367d0c0-7f61-4da6-a376-c03cbbe9b5ce None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:35.144347 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367d0c0-7f61-4da6-a376-c03cbbe9b5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:35.145115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367d0c0-7f61-4da6-a376-c03cbbe9b5ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:35.227493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56002d50-64d4-4662-9b86-7f33edc3fec4 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:35.329844 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd78b9-7f06-4f47-b2be-b95822211912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:35.330931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd78b9-7f06-4f47-b2be-b95822211912 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:35.331410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd78b9-7f06-4f47-b2be-b95822211912 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:35.331902 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd78b9-7f06-4f47-b2be-b95822211912 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:35.333095 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd78b9-7f06-4f47-b2be-b95822211912 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:35.333646 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd78b9-7f06-4f47-b2be-b95822211912 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:35.341058 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd78b9-7f06-4f47-b2be-b95822211912 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:35.372481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 215/423] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:00:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:35.382117 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367d0c0-7f61-4da6-a376-c03cbbe9b5ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:35.384860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367d0c0-7f61-4da6-a376-c03cbbe9b5ce None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:35.390005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd78b9-7f06-4f47-b2be-b95822211912 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:35.483606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:35.612381 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd78b9-7f06-4f47-b2be-b95822211912 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:35.657970 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4acbd-62ec-4b74-b091-1e6bf9f35536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:35.658736 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4acbd-62ec-4b74-b091-1e6bf9f35536 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:35.659701 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4acbd-62ec-4b74-b091-1e6bf9f35536 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:35.660069 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4acbd-62ec-4b74-b091-1e6bf9f35536 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:35.686839 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 209/424] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 920 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 16:00:35.695528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73179d30-758e-4b58-935b-5b93ad1ae14a None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:35.695528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73179d30-758e-4b58-935b-5b93ad1ae14a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:35.695528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73179d30-758e-4b58-935b-5b93ad1ae14a None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:35.695528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 210/425] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:35.701388 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d9296-a5fa-4e9a-86b1-f15e562e8513 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:35.702218 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d9296-a5fa-4e9a-86b1-f15e562e8513 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:35.702924 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d9296-a5fa-4e9a-86b1-f15e562e8513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:35.750088 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8d2d9296-a5fa-4e9a-86b1-f15e562e8513 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:35.751324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8d2d9296-a5fa-4e9a-86b1-f15e562e8513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:35.779322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:35.791058 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4acbd-62ec-4b74-b091-1e6bf9f35536 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:35.795833 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4acbd-62ec-4b74-b091-1e6bf9f35536 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:35.843768 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 211/426] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:35.846021 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d9296-a5fa-4e9a-86b1-f15e562e8513 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:35.863543 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:36.068275 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 216/427] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 16:00:36.069656 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4acbd-62ec-4b74-b091-1e6bf9f35536 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:36.099274 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d9296-a5fa-4e9a-86b1-f15e562e8513 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:36.201795 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324a94b-511f-4158-8a53-e9c2ed6aeca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:36.202464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324a94b-511f-4158-8a53-e9c2ed6aeca3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:36.202768 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324a94b-511f-4158-8a53-e9c2ed6aeca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:36.203032 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324a94b-511f-4158-8a53-e9c2ed6aeca3 None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:36.204621 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324a94b-511f-4158-8a53-e9c2ed6aeca3 None admin] RBAC: Authorizing `identity:get_project(project_id=dda7941c8f444d41afc018b93fcd36fe)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:36.206206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324a94b-511f-4158-8a53-e9c2ed6aeca3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:36.209756 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 212/428] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:35 2022] GET /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe => generated 298 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:36.216830 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324a94b-511f-4158-8a53-e9c2ed6aeca3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:36.232631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4acbd-62ec-4b74-b091-1e6bf9f35536 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:36.244855 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:36.285582 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:36.302231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc4a8a-ee37-4da0-850a-2b1436ac64fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:36.304272 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc4a8a-ee37-4da0-850a-2b1436ac64fd None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:36.305023 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc4a8a-ee37-4da0-850a-2b1436ac64fd None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:36.305917 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc4a8a-ee37-4da0-850a-2b1436ac64fd None admin] PATH_INFO: `/v3/projects/dda7941c8f444d41afc018b93fcd36fe` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:36.308693 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc4a8a-ee37-4da0-850a-2b1436ac64fd None admin] RBAC: Authorizing `identity:get_project(project_id=dda7941c8f444d41afc018b93fcd36fe)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:36.312164 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc4a8a-ee37-4da0-850a-2b1436ac64fd None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:36.316192 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 217/429] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:36 2022] GET /identity/v3/projects/dda7941c8f444d41afc018b93fcd36fe => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:36.317914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc4a8a-ee37-4da0-850a-2b1436ac64fd None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:36.348903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324a94b-511f-4158-8a53-e9c2ed6aeca3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:36.396169 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead74e4-b236-4399-bf63-92eec75f6f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:36.397456 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead74e4-b236-4399-bf63-92eec75f6f8a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:36.397729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead74e4-b236-4399-bf63-92eec75f6f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:36.398000 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead74e4-b236-4399-bf63-92eec75f6f8a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:36.412311 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead74e4-b236-4399-bf63-92eec75f6f8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:36.419130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-bead74e4-b236-4399-bf63-92eec75f6f8a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:00:36.419130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:36.419130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:36.419130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:36.419130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:36.419130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:36.419130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:36.419130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:36.419130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:36.419130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:36.419130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:36.419130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:36.419130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:36.419130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:36.419130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:36.419130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:36.420565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:36.420565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:36.420565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:36.420565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:36.420565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:36.420565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:36.420565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:36.420565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:36.420565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:36.420565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:36.420565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:36.420565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:36.420565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:36.420565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:36.420565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:36.420565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:36.422155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:36.422155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:36.422155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:36.422155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:36.422155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:36.422155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:36.422155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:36.422155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:36.422155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:36.422155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:36.422155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:00:36.422155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:36.422155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 213/430] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 16:00:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:36.422155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead74e4-b236-4399-bf63-92eec75f6f8a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:36.530064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc4a8a-ee37-4da0-850a-2b1436ac64fd None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:36.531502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:36.557392 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead74e4-b236-4399-bf63-92eec75f6f8a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:36.603712 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b1904-95e6-40f0-9859-eeb59ba39513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:36.604260 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b1904-95e6-40f0-9859-eeb59ba39513 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:36.604534 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b1904-95e6-40f0-9859-eeb59ba39513 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:36.604803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b1904-95e6-40f0-9859-eeb59ba39513 None admin] PATH_INFO: `/v3/groups/4c446fe47fe64d38baacd39ac1b11320` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:36.606507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b1904-95e6-40f0-9859-eeb59ba39513 None admin] RBAC: Authorizing `identity:get_group(group_id=4c446fe47fe64d38baacd39ac1b11320)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:36.608396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b1904-95e6-40f0-9859-eeb59ba39513 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:36.618356 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 214/431] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 6 16:00:36 2022] GET /identity/v3/groups/4c446fe47fe64d38baacd39ac1b11320 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:36.618356 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b1904-95e6-40f0-9859-eeb59ba39513 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:36.622677 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b75597-fc36-4d47-b738-f030f8e07ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:36.624145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b75597-fc36-4d47-b738-f030f8e07ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:36.624894 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b75597-fc36-4d47-b738-f030f8e07ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:36.625623 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b75597-fc36-4d47-b738-f030f8e07ba5 None admin] PATH_INFO: `/v3/groups/4c446fe47fe64d38baacd39ac1b11320` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:36.628122 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b75597-fc36-4d47-b738-f030f8e07ba5 None admin] RBAC: Authorizing `identity:get_group(group_id=4c446fe47fe64d38baacd39ac1b11320)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:36.631882 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b75597-fc36-4d47-b738-f030f8e07ba5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:36.642476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 218/432] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 6 16:00:36 2022] GET /identity/v3/groups/4c446fe47fe64d38baacd39ac1b11320 => generated 226 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:36.644563 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b75597-fc36-4d47-b738-f030f8e07ba5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:36.747433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:36.772376 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b1904-95e6-40f0-9859-eeb59ba39513 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:36.798589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b75597-fc36-4d47-b738-f030f8e07ba5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:36.846926 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4e1ba-c6da-42c8-a6bd-1e7ca132d784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:36.847542 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4e1ba-c6da-42c8-a6bd-1e7ca132d784 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:36.847817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4e1ba-c6da-42c8-a6bd-1e7ca132d784 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:36.848085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4e1ba-c6da-42c8-a6bd-1e7ca132d784 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:36.848818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4e1ba-c6da-42c8-a6bd-1e7ca132d784 None admin] RBAC: Adding query filter params (group.id=4c446fe47fe64d38baacd39ac1b11320, role.id=bad9ff35005240e99bedef3fddb1dc77, scope.project.id=dda7941c8f444d41afc018b93fcd36fe) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:36.849125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4e1ba-c6da-42c8-a6bd-1e7ca132d784 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:36.849891 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac1355-0d3e-4dbe-b2b8-671d51eaf8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:36.850723 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4e1ba-c6da-42c8-a6bd-1e7ca132d784 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:36.851566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac1355-0d3e-4dbe-b2b8-671d51eaf8a8 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:36.852407 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac1355-0d3e-4dbe-b2b8-671d51eaf8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:36.853310 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac1355-0d3e-4dbe-b2b8-671d51eaf8a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:36.855110 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac1355-0d3e-4dbe-b2b8-671d51eaf8a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:36.855971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac1355-0d3e-4dbe-b2b8-671d51eaf8a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:36.866700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 219/433] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 6 16:00:36 2022] GET /identity/v3/role_assignments?group.id=4c446fe47fe64d38baacd39ac1b11320&scope.project.id=dda7941c8f444d41afc018b93fcd36fe&role.id=bad9ff35005240e99bedef3fddb1dc77 => generated 611 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:36.866700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4e1ba-c6da-42c8-a6bd-1e7ca132d784 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:36.867628 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac1355-0d3e-4dbe-b2b8-671d51eaf8a8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:36.887913 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 215/434] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:00:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:36.897895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac1355-0d3e-4dbe-b2b8-671d51eaf8a8 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:36.933635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:37.060190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac1355-0d3e-4dbe-b2b8-671d51eaf8a8 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:37.086647 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4e1ba-c6da-42c8-a6bd-1e7ca132d784 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:37.117101 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2074b2-1154-4808-bf15-ab0a570002cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:37.118094 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2074b2-1154-4808-bf15-ab0a570002cf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:37.118419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2074b2-1154-4808-bf15-ab0a570002cf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:37.118807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2074b2-1154-4808-bf15-ab0a570002cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:37.121323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:37.134035 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3b765-794e-4eb4-bbe2-4fae3917361c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:37.134837 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3b765-794e-4eb4-bbe2-4fae3917361c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:37.135418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3b765-794e-4eb4-bbe2-4fae3917361c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:37.135953 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3b765-794e-4eb4-bbe2-4fae3917361c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:37.138033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3b765-794e-4eb4-bbe2-4fae3917361c None admin] RBAC: Adding query filter params (group.id=4c446fe47fe64d38baacd39ac1b11320, role.id=e89162686187404da84651e0a3632b7b, scope.project.id=dda7941c8f444d41afc018b93fcd36fe) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:37.138403 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3b765-794e-4eb4-bbe2-4fae3917361c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:37.140271 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3b765-794e-4eb4-bbe2-4fae3917361c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:37.157786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 220/435] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 6 16:00:36 2022] GET /identity/v3/role_assignments?group.id=4c446fe47fe64d38baacd39ac1b11320&scope.project.id=dda7941c8f444d41afc018b93fcd36fe&role.id=e89162686187404da84651e0a3632b7b => generated 611 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:37.157786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befd38d-5131-4728-b42c-b6cdcab778ea None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:37.157786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befd38d-5131-4728-b42c-b6cdcab778ea None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:37.157786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befd38d-5131-4728-b42c-b6cdcab778ea None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:37.157786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 221/436] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:37.158599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6ba6b-3db5-4db5-974e-de67b403c937 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:37.159057 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6ba6b-3db5-4db5-974e-de67b403c937 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:37.159331 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6ba6b-3db5-4db5-974e-de67b403c937 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:37.161055 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 222/437] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:37.172742 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0804a08-6a29-40b5-b897-b504b733a625 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:37.173183 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0804a08-6a29-40b5-b897-b504b733a625 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:37.173639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0804a08-6a29-40b5-b897-b504b733a625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:37.199280 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d0804a08-6a29-40b5-b897-b504b733a625 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:37.201028 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d0804a08-6a29-40b5-b897-b504b733a625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:37.236544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2074b2-1154-4808-bf15-ab0a570002cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:37.238066 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2074b2-1154-4808-bf15-ab0a570002cf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:37.261133 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:37.300469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 223/438] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:37.300469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bf348-604e-4ff7-9202-30d0d3cb18eb None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:37.300469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bf348-604e-4ff7-9202-30d0d3cb18eb None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:37.300469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bf348-604e-4ff7-9202-30d0d3cb18eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:37.317104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-fe8bf348-604e-4ff7-9202-30d0d3cb18eb None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:37.318912 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fe8bf348-604e-4ff7-9202-30d0d3cb18eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:37.406252 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 216/439] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 16:00:37.413611 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2074b2-1154-4808-bf15-ab0a570002cf None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:37.417297 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 224/440] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:37.590139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8bf348-604e-4ff7-9202-30d0d3cb18eb None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:37.593022 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2074b2-1154-4808-bf15-ab0a570002cf None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:37.624567 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:37.667097 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9eaff-c44d-4447-a6c7-448d4ba9e370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:37.667660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9eaff-c44d-4447-a6c7-448d4ba9e370 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:37.667936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9eaff-c44d-4447-a6c7-448d4ba9e370 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:37.668204 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9eaff-c44d-4447-a6c7-448d4ba9e370 None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:37.669975 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed9eaff-c44d-4447-a6c7-448d4ba9e370 None admin] RBAC: Authorizing `identity:get_project(project_id=c57cd5c32d1447d3968cda9fc409de3c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:37.671533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed9eaff-c44d-4447-a6c7-448d4ba9e370 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:37.673235 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 217/441] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:37 2022] GET /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:37.679595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9eaff-c44d-4447-a6c7-448d4ba9e370 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:37.680047 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:37.790847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8bf348-604e-4ff7-9202-30d0d3cb18eb None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:37.827262 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9eaff-c44d-4447-a6c7-448d4ba9e370 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:37.856627 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f93f5-053c-48a5-95e7-7f6f64e620a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:37.857851 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f93f5-053c-48a5-95e7-7f6f64e620a2 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:37.858431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f93f5-053c-48a5-95e7-7f6f64e620a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:37.858956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f93f5-053c-48a5-95e7-7f6f64e620a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:37.879793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f93f5-053c-48a5-95e7-7f6f64e620a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:37.885584 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0d875-e332-4688-a313-7937104fda42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:37.885873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-305f93f5-053c-48a5-95e7-7f6f64e620a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:37.885873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:37.885873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:37.885873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:37.885873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:37.885873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:37.885873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:37.885873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:37.885873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:37.885873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:37.885873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:37.885873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:37.885873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:37.885873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:37.885873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:37.885873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:37.887807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:37.887807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:37.887807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:37.887807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:37.887807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:37.887807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:37.887807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:37.887807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:37.887807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:37.887807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:37.887807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:37.887807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:37.887807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:37.887807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:37.887807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:37.887807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:37.890223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:37.890223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:37.890223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:37.890223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:37.890223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:37.890223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:37.890223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:37.890223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:37.890223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:37.890223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:37.890223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 16:00:37.890223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:37.890223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 225/442] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:00:37.892229 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0d875-e332-4688-a313-7937104fda42 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:37.892371 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f93f5-053c-48a5-95e7-7f6f64e620a2 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:37.892513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0d875-e332-4688-a313-7937104fda42 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:37.892513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0d875-e332-4688-a313-7937104fda42 None admin] PATH_INFO: `/v3/groups/4c446fe47fe64d38baacd39ac1b11320` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:37.892513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0d875-e332-4688-a313-7937104fda42 None admin] RBAC: Authorizing `identity:get_group(group_id=4c446fe47fe64d38baacd39ac1b11320)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:37.894718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0d875-e332-4688-a313-7937104fda42 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:37.902880 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 218/443] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 6 16:00:37 2022] GET /identity/v3/groups/4c446fe47fe64d38baacd39ac1b11320 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:37.904618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0d875-e332-4688-a313-7937104fda42 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:38.005479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:38.054510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f93f5-053c-48a5-95e7-7f6f64e620a2 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:38.071183 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:38.108529 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eac756-49ca-437f-88d6-d63022a74e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:38.109532 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eac756-49ca-437f-88d6-d63022a74e1e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:38.111164 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eac756-49ca-437f-88d6-d63022a74e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:38.111624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eac756-49ca-437f-88d6-d63022a74e1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:38.115036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0d875-e332-4688-a313-7937104fda42 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:38.129511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eac756-49ca-437f-88d6-d63022a74e1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:38.138846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-69eac756-49ca-437f-88d6-d63022a74e1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:00:38.138846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:38.138846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:38.138846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:38.138846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:38.138846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:38.138846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:38.138846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:38.138846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:38.138846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:38.138846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:38.138846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:38.138846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:38.138846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:38.138846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:38.138846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:38.139821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:38.139821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:38.139821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:38.139821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:38.139821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:38.139821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:38.139821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:38.139821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:38.139821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:38.139821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:38.139821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:38.139821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:38.139821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:38.139821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:38.139821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:38.139821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:38.140587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:38.140587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:38.140587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:38.140587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:38.140587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:38.140587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:38.140587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:38.140587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:38.140587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:38.140587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:38.140587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:00:38.140587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:38.141185 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 226/444] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:00:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:38.141847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eac756-49ca-437f-88d6-d63022a74e1e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:38.170071 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85154081-1e6e-44d1-b428-02adee6bc663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:38.170706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85154081-1e6e-44d1-b428-02adee6bc663 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:38.170993 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85154081-1e6e-44d1-b428-02adee6bc663 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:38.171263 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85154081-1e6e-44d1-b428-02adee6bc663 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:38.171935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85154081-1e6e-44d1-b428-02adee6bc663 None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:38.172241 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85154081-1e6e-44d1-b428-02adee6bc663 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:38.173650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85154081-1e6e-44d1-b428-02adee6bc663 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:38.183723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 219/445] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 16:00:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:38.184579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85154081-1e6e-44d1-b428-02adee6bc663 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:38.276788 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eac756-49ca-437f-88d6-d63022a74e1e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:38.320454 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848f3d3-0712-45da-9018-2f3b9a2be6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:38.321351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848f3d3-0712-45da-9018-2f3b9a2be6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:38.321786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848f3d3-0712-45da-9018-2f3b9a2be6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:38.322272 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848f3d3-0712-45da-9018-2f3b9a2be6f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:38.323687 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848f3d3-0712-45da-9018-2f3b9a2be6f9 None admin] RBAC: Adding query filter params (group.id=4c446fe47fe64d38baacd39ac1b11320, role.id=bad9ff35005240e99bedef3fddb1dc77, scope.project.id=c57cd5c32d1447d3968cda9fc409de3c) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:38.324243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848f3d3-0712-45da-9018-2f3b9a2be6f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:38.327305 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848f3d3-0712-45da-9018-2f3b9a2be6f9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:38.336731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 227/446] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 6 16:00:38 2022] GET /identity/v3/role_assignments?group.id=4c446fe47fe64d38baacd39ac1b11320&scope.project.id=c57cd5c32d1447d3968cda9fc409de3c&role.id=bad9ff35005240e99bedef3fddb1dc77 => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:38.337186 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85154081-1e6e-44d1-b428-02adee6bc663 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:38.337580 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848f3d3-0712-45da-9018-2f3b9a2be6f9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:38.371248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:38.383423 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:38.403817 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56542782-9afe-42aa-9bf4-81fcd1ccf16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:38.404438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56542782-9afe-42aa-9bf4-81fcd1ccf16b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:38.404719 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56542782-9afe-42aa-9bf4-81fcd1ccf16b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:38.404966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56542782-9afe-42aa-9bf4-81fcd1ccf16b None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:38.405729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56542782-9afe-42aa-9bf4-81fcd1ccf16b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:38.405980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56542782-9afe-42aa-9bf4-81fcd1ccf16b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:38.407498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56542782-9afe-42aa-9bf4-81fcd1ccf16b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:38.418296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 220/447] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:38.424141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56542782-9afe-42aa-9bf4-81fcd1ccf16b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:38.506611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848f3d3-0712-45da-9018-2f3b9a2be6f9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:38.572819 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357f18f-d3b7-405c-92c7-9865f60f133d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:38.574082 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357f18f-d3b7-405c-92c7-9865f60f133d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:38.574551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357f18f-d3b7-405c-92c7-9865f60f133d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:38.576836 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357f18f-d3b7-405c-92c7-9865f60f133d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:38.591104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56542782-9afe-42aa-9bf4-81fcd1ccf16b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:38.638616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02130da-bc50-42d3-8f37-e6958bae5ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:38.639293 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02130da-bc50-42d3-8f37-e6958bae5ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:38.639597 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02130da-bc50-42d3-8f37-e6958bae5ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:38.640044 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02130da-bc50-42d3-8f37-e6958bae5ca6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:38.656936 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357f18f-d3b7-405c-92c7-9865f60f133d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:38.658552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357f18f-d3b7-405c-92c7-9865f60f133d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:38.689449 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:38.695151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:38.718430 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02130da-bc50-42d3-8f37-e6958bae5ca6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:38.719985 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02130da-bc50-42d3-8f37-e6958bae5ca6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:38.767509 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 228/448] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:00:38.773310 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357f18f-d3b7-405c-92c7-9865f60f133d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:38.827100 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 221/449] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 16:00:38.832732 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02130da-bc50-42d3-8f37-e6958bae5ca6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:38.903703 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357f18f-d3b7-405c-92c7-9865f60f133d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:38.921751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:38.944666 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b8e33-416f-4bb6-8e76-2f60e7f7d67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:38.945573 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b8e33-416f-4bb6-8e76-2f60e7f7d67c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:38.945573 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b8e33-416f-4bb6-8e76-2f60e7f7d67c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:38.945872 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b8e33-416f-4bb6-8e76-2f60e7f7d67c None admin] PATH_INFO: `/v3/projects/c57cd5c32d1447d3968cda9fc409de3c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:38.947011 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b8e33-416f-4bb6-8e76-2f60e7f7d67c None admin] RBAC: Authorizing `identity:get_project(project_id=c57cd5c32d1447d3968cda9fc409de3c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:38.948465 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b8e33-416f-4bb6-8e76-2f60e7f7d67c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:38.949986 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02130da-bc50-42d3-8f37-e6958bae5ca6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:38.950119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 229/450] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:38 2022] GET /identity/v3/projects/c57cd5c32d1447d3968cda9fc409de3c => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:38.955902 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b8e33-416f-4bb6-8e76-2f60e7f7d67c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:38.960032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:38.989492 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91927e8-6b9b-46cf-bfa2-a88aab5824e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:38.990186 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91927e8-6b9b-46cf-bfa2-a88aab5824e7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:38.990985 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91927e8-6b9b-46cf-bfa2-a88aab5824e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:38.991577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91927e8-6b9b-46cf-bfa2-a88aab5824e7 None admin] PATH_INFO: `/v3/projects/dcbe5a1252a54f0a914b3779a09b044d` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:38.992427 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91927e8-6b9b-46cf-bfa2-a88aab5824e7 None admin] RBAC: Authorizing `identity:get_project(project_id=dcbe5a1252a54f0a914b3779a09b044d)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:38.993860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91927e8-6b9b-46cf-bfa2-a88aab5824e7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:38.995582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 222/451] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:00:38 2022] GET /identity/v3/projects/dcbe5a1252a54f0a914b3779a09b044d => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:39.001495 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91927e8-6b9b-46cf-bfa2-a88aab5824e7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:39.064829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b8e33-416f-4bb6-8e76-2f60e7f7d67c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:39.101827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c2708-67fc-4d39-a1b8-ca2689d55acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:39.102456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c2708-67fc-4d39-a1b8-ca2689d55acd None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:39.102760 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c2708-67fc-4d39-a1b8-ca2689d55acd None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:39.103032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c2708-67fc-4d39-a1b8-ca2689d55acd None admin] PATH_INFO: `/v3/groups/4c446fe47fe64d38baacd39ac1b11320` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:39.104181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c2708-67fc-4d39-a1b8-ca2689d55acd None admin] RBAC: Authorizing `identity:get_group(group_id=4c446fe47fe64d38baacd39ac1b11320)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:39.105619 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c2708-67fc-4d39-a1b8-ca2689d55acd None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:39.107243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 230/452] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 6 16:00:38 2022] GET /identity/v3/groups/4c446fe47fe64d38baacd39ac1b11320 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:39.113161 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c2708-67fc-4d39-a1b8-ca2689d55acd None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:39.120568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91927e8-6b9b-46cf-bfa2-a88aab5824e7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:39.163087 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3544515-a0de-4313-8e54-87e902026b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:39.163779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3544515-a0de-4313-8e54-87e902026b4a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:39.164055 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3544515-a0de-4313-8e54-87e902026b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:39.164324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3544515-a0de-4313-8e54-87e902026b4a None admin] PATH_INFO: `/v3/groups/324b3694e04548d698dc3f058c0ccd69` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:39.165746 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3544515-a0de-4313-8e54-87e902026b4a None admin] RBAC: Authorizing `identity:get_group(group_id=324b3694e04548d698dc3f058c0ccd69)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:39.167435 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3544515-a0de-4313-8e54-87e902026b4a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:39.169029 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 223/453] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 6 16:00:39 2022] GET /identity/v3/groups/324b3694e04548d698dc3f058c0ccd69 => generated 229 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:39.175072 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3544515-a0de-4313-8e54-87e902026b4a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:39.188130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:39.191298 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:39.225952 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c2708-67fc-4d39-a1b8-ca2689d55acd None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:39.264062 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868033cd-43c4-4621-96eb-8c2774bcab79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:39.264697 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868033cd-43c4-4621-96eb-8c2774bcab79 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:39.264973 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868033cd-43c4-4621-96eb-8c2774bcab79 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:39.265244 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868033cd-43c4-4621-96eb-8c2774bcab79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:39.265968 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868033cd-43c4-4621-96eb-8c2774bcab79 None admin] RBAC: Adding query filter params (group.id=4c446fe47fe64d38baacd39ac1b11320, role.id=e89162686187404da84651e0a3632b7b, scope.project.id=c57cd5c32d1447d3968cda9fc409de3c) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:39.266275 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868033cd-43c4-4621-96eb-8c2774bcab79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:39.267754 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868033cd-43c4-4621-96eb-8c2774bcab79 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:39.274683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 231/454] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 6 16:00:39 2022] GET /identity/v3/role_assignments?group.id=4c446fe47fe64d38baacd39ac1b11320&scope.project.id=c57cd5c32d1447d3968cda9fc409de3c&role.id=e89162686187404da84651e0a3632b7b => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:39.297471 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3544515-a0de-4313-8e54-87e902026b4a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:39.345212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9a0d4-0201-4c10-b83d-5056b349f194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:39.345773 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9a0d4-0201-4c10-b83d-5056b349f194 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:39.346052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9a0d4-0201-4c10-b83d-5056b349f194 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:39.346323 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9a0d4-0201-4c10-b83d-5056b349f194 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:39.347109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9a0d4-0201-4c10-b83d-5056b349f194 None admin] RBAC: Adding query filter params (group.id=324b3694e04548d698dc3f058c0ccd69, role.id=ba05677650e046a48d5f19440d48fbf2, scope.project.id=dcbe5a1252a54f0a914b3779a09b044d) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:39.347420 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9a0d4-0201-4c10-b83d-5056b349f194 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:39.348871 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9a0d4-0201-4c10-b83d-5056b349f194 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:39.356437 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 224/455] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 6 16:00:39 2022] GET /identity/v3/role_assignments?group.id=324b3694e04548d698dc3f058c0ccd69&scope.project.id=dcbe5a1252a54f0a914b3779a09b044d&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:42.377653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c77855-b4bd-4919-84ba-31e8adfeb1ed None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:42.378697 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c77855-b4bd-4919-84ba-31e8adfeb1ed None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:42.379176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c77855-b4bd-4919-84ba-31e8adfeb1ed None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:42.381507 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 232/456] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:42.388253 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c194eb-7827-411e-b128-6aabd5f61a44 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:42.389128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c194eb-7827-411e-b128-6aabd5f61a44 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:42.389592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c194eb-7827-411e-b128-6aabd5f61a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:42.412875 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-59c194eb-7827-411e-b128-6aabd5f61a44 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:42.414408 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-59c194eb-7827-411e-b128-6aabd5f61a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:42.442896 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bead96-0df3-46fa-9eb6-76802a66b836 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:42.443740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bead96-0df3-46fa-9eb6-76802a66b836 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:42.444175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bead96-0df3-46fa-9eb6-76802a66b836 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:42.445874 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 233/457] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:42.452367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af8b0a-f024-4b8e-9dc8-119362cee807 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:42.452853 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af8b0a-f024-4b8e-9dc8-119362cee807 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:42.453332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af8b0a-f024-4b8e-9dc8-119362cee807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:42.491672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-40af8b0a-f024-4b8e-9dc8-119362cee807 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:42.493306 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-40af8b0a-f024-4b8e-9dc8-119362cee807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:42.526496 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 225/458] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:42.528216 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230de1b8-cf3b-44d5-8e6e-75ce52ad5eda None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:42.528703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230de1b8-cf3b-44d5-8e6e-75ce52ad5eda None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:42.529166 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230de1b8-cf3b-44d5-8e6e-75ce52ad5eda None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:42.530965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 226/459] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:42.536485 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683d8d5-0a81-4f8a-8c8d-2935e86d8568 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:42.537578 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683d8d5-0a81-4f8a-8c8d-2935e86d8568 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:42.538046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683d8d5-0a81-4f8a-8c8d-2935e86d8568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:42.550700 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:42.596371 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d683d8d5-0a81-4f8a-8c8d-2935e86d8568 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:42.598579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d683d8d5-0a81-4f8a-8c8d-2935e86d8568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:42.619344 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 234/460] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:42.753444 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 227/461] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:00:42.913538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af8b0a-f024-4b8e-9dc8-119362cee807 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:43.003369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:43.048235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af8b0a-f024-4b8e-9dc8-119362cee807 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:43.089462 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bfa421-b28a-48f7-b32d-9e9bb3d911d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:43.090145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfa421-b28a-48f7-b32d-9e9bb3d911d9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:43.090472 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfa421-b28a-48f7-b32d-9e9bb3d911d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:43.090777 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfa421-b28a-48f7-b32d-9e9bb3d911d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:43.103411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bfa421-b28a-48f7-b32d-9e9bb3d911d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:43.105030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bfa421-b28a-48f7-b32d-9e9bb3d911d9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:43.123470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-e4bfa421-b28a-48f7-b32d-9e9bb3d911d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:43.123470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:43.123470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:43.123470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:43.123470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:43.123470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:43.123470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:43.123470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:43.123470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:43.123470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:43.123470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:43.123470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:43.123470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:00:43.123470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:00:43.123470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:00:43.123470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:00:43.125067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:43.125067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:43.125067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:43.125067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:43.125067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:43.125067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:43.125067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:43.125067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:43.125067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:43.125067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:43.125067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:43.125067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:43.125067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:43.125067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:43.125067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:43.125067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:43.125067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:00:43.126724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:00:43.126724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:43.126724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:43.126724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 235/462] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:43.132185 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683d8d5-0a81-4f8a-8c8d-2935e86d8568 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:43.206716 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bfa421-b28a-48f7-b32d-9e9bb3d911d9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:43.357687 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683d8d5-0a81-4f8a-8c8d-2935e86d8568 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:43.418523 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bfa421-b28a-48f7-b32d-9e9bb3d911d9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:43.439372 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:43.440153 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d4ce2-ffec-4f42-a1ff-2fea1d4b6f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:43.441881 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d4ce2-ffec-4f42-a1ff-2fea1d4b6f1c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:43.443773 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d4ce2-ffec-4f42-a1ff-2fea1d4b6f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:43.444506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d4ce2-ffec-4f42-a1ff-2fea1d4b6f1c None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:43.446535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d4ce2-ffec-4f42-a1ff-2fea1d4b6f1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:43.447642 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d4ce2-ffec-4f42-a1ff-2fea1d4b6f1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:43.451033 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d4ce2-ffec-4f42-a1ff-2fea1d4b6f1c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:43.454501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:43.461515 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17599080-4a47-48d4-9a30-72c8d6c5f1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:43.462205 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17599080-4a47-48d4-9a30-72c8d6c5f1bd None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:43.462480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17599080-4a47-48d4-9a30-72c8d6c5f1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:43.462834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17599080-4a47-48d4-9a30-72c8d6c5f1bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:43.476335 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17599080-4a47-48d4-9a30-72c8d6c5f1bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:43.477850 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17599080-4a47-48d4-9a30-72c8d6c5f1bd None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:43.487274 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 228/463] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:43.487274 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d4ce2-ffec-4f42-a1ff-2fea1d4b6f1c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:43.500908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-17599080-4a47-48d4-9a30-72c8d6c5f1bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:43.500908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:43.500908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:43.500908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:43.500908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:43.500908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:43.500908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:43.500908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:43.500908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:43.500908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:43.500908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:43.500908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:43.500908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:00:43.500908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:00:43.500908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:00:43.500908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:00:43.502060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:43.502060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:43.502060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:43.502060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:43.502060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:43.502060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:43.502060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:43.502060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:43.502060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:43.502060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:43.502060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:43.502060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:43.502060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:43.502060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:43.502060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:43.502060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:43.502060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:00:43.503126 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:00:43.503126 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:43.503126 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:43.504842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 236/464] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:00:43.506323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17599080-4a47-48d4-9a30-72c8d6c5f1bd None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:43.667440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17599080-4a47-48d4-9a30-72c8d6c5f1bd None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:43.673145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d4ce2-ffec-4f42-a1ff-2fea1d4b6f1c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:43.714532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc05c29-f3e3-4f98-b682-dd45440934bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:43.715206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc05c29-f3e3-4f98-b682-dd45440934bc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:43.715507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc05c29-f3e3-4f98-b682-dd45440934bc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:43.715735 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc05c29-f3e3-4f98-b682-dd45440934bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:43.729202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc05c29-f3e3-4f98-b682-dd45440934bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:43.730691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc05c29-f3e3-4f98-b682-dd45440934bc None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:43.733714 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57fd5d-11e3-4a21-a49f-4b4823d5b823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:43.734795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57fd5d-11e3-4a21-a49f-4b4823d5b823 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:43.735284 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57fd5d-11e3-4a21-a49f-4b4823d5b823 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:43.735767 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57fd5d-11e3-4a21-a49f-4b4823d5b823 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:43.736975 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57fd5d-11e3-4a21-a49f-4b4823d5b823 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:43.739779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57fd5d-11e3-4a21-a49f-4b4823d5b823 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:43.747601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-bbc05c29-f3e3-4f98-b682-dd45440934bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:43.747601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:43.747601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:43.747601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:43.747601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:43.747601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:43.747601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:43.747601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:43.747601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:43.747601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:43.747601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:43.747601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:43.747601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:00:43.747601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:00:43.747601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:00:43.747601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:00:43.756102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:43.756102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:43.756102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:43.756102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:43.756102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:43.756102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:43.756102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:43.756102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:43.756102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:43.756102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:43.756102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:43.756102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:43.756102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:43.756102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:43.756102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:43.756102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:43.756102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:00:43.758418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:00:43.758418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:43.758418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:43.758418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 229/465] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:43.758418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98041f-eb99-4058-874d-fb6dc1ab9536 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:43.758418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98041f-eb99-4058-874d-fb6dc1ab9536 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:43.758418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98041f-eb99-4058-874d-fb6dc1ab9536 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:43.758418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 230/466] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:43.758418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98041f-eb99-4058-874d-fb6dc1ab9536 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:43.794698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 237/467] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 6 16:00:43 2022] POST /identity/v3/users => generated 246 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:00:43.794698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57fd5d-11e3-4a21-a49f-4b4823d5b823 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:43.838088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:43.859712 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:43.963759 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57fd5d-11e3-4a21-a49f-4b4823d5b823 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:43.964909 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98041f-eb99-4058-874d-fb6dc1ab9536 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:44.008358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f750e-7b66-4790-abcf-e07ba71fa3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:44.009080 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f750e-7b66-4790-abcf-e07ba71fa3da None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:44.009457 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f750e-7b66-4790-abcf-e07ba71fa3da None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:44.009773 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f750e-7b66-4790-abcf-e07ba71fa3da None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:44.010758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f750e-7b66-4790-abcf-e07ba71fa3da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:44.011144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f750e-7b66-4790-abcf-e07ba71fa3da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:44.012831 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f750e-7b66-4790-abcf-e07ba71fa3da None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:44.030905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 231/468] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:44.033367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae2c73-7d21-456a-aadb-ef51ce8ea447 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:44.033869 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae2c73-7d21-456a-aadb-ef51ce8ea447 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:44.034333 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae2c73-7d21-456a-aadb-ef51ce8ea447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:44.038748 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d089f-6184-49dd-a045-0db6527c11be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:44.039372 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d089f-6184-49dd-a045-0db6527c11be None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:44.039759 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d089f-6184-49dd-a045-0db6527c11be None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:44.040041 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d089f-6184-49dd-a045-0db6527c11be None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:44.040808 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d089f-6184-49dd-a045-0db6527c11be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:44.041209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d089f-6184-49dd-a045-0db6527c11be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:44.042725 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d089f-6184-49dd-a045-0db6527c11be None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:44.067858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 238/469] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:44.070953 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d089f-6184-49dd-a045-0db6527c11be None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:44.073852 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a7ae2c73-7d21-456a-aadb-ef51ce8ea447 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:44.077236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a7ae2c73-7d21-456a-aadb-ef51ce8ea447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:44.213194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 232/470] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:44.218929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae2c73-7d21-456a-aadb-ef51ce8ea447 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:44.252836 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:44.270483 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d089f-6184-49dd-a045-0db6527c11be None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:44.344926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:44.359012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea03554-490d-4f21-817c-824cd563108d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:44.360059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea03554-490d-4f21-817c-824cd563108d None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:44.360541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea03554-490d-4f21-817c-824cd563108d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:44.361021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea03554-490d-4f21-817c-824cd563108d None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:44.362242 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea03554-490d-4f21-817c-824cd563108d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:44.367653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea03554-490d-4f21-817c-824cd563108d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:44.436348 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 239/471] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 6 16:00:44 2022] POST /identity/v3/users => generated 248 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:00:44.531604 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae2c73-7d21-456a-aadb-ef51ce8ea447 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:44.612586 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130af3b5-6d05-4198-b34a-1cb0a86fc608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:44.613220 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130af3b5-6d05-4198-b34a-1cb0a86fc608 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:44.613507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130af3b5-6d05-4198-b34a-1cb0a86fc608 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:44.613868 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130af3b5-6d05-4198-b34a-1cb0a86fc608 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:44.614548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130af3b5-6d05-4198-b34a-1cb0a86fc608 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:44.616143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130af3b5-6d05-4198-b34a-1cb0a86fc608 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:44.653892 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 233/472] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 6 16:00:44 2022] POST /identity/v3/users => generated 248 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:00:44.818603 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea03554-490d-4f21-817c-824cd563108d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:44.945694 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f3b32-373c-47f2-a8ff-d962d2b61ee0 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:44.946186 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f3b32-373c-47f2-a8ff-d962d2b61ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:44.946631 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f3b32-373c-47f2-a8ff-d962d2b61ee0 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:44.955163 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 234/473] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:44 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:44.960509 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf3184-35c2-43b7-9acd-e77bee92b7b0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:44.960970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf3184-35c2-43b7-9acd-e77bee92b7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:44.961451 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf3184-35c2-43b7-9acd-e77bee92b7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:44.970983 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea03554-490d-4f21-817c-824cd563108d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:45.008949 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-34cf3184-35c2-43b7-9acd-e77bee92b7b0 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:45.011821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-34cf3184-35c2-43b7-9acd-e77bee92b7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:45.033101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36acf8e9-d244-4957-972a-a51e62966666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:45.033752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acf8e9-d244-4957-972a-a51e62966666 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:45.034020 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acf8e9-d244-4957-972a-a51e62966666 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:45.034280 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acf8e9-d244-4957-972a-a51e62966666 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:45.046445 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36acf8e9-d244-4957-972a-a51e62966666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:45.048180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36acf8e9-d244-4957-972a-a51e62966666 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:45.051176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:45.068966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-36acf8e9-d244-4957-972a-a51e62966666 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:45.068966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:45.068966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:45.068966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:45.068966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:45.068966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:45.068966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:45.068966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:45.068966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:45.068966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:45.068966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:45.068966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:45.068966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:00:45.068966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:00:45.068966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:00:45.068966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:00:45.070820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:45.070820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:45.070820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:45.070820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:45.070820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:45.070820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:45.070820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:45.070820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:45.070820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:45.070820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:45.070820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:45.070820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:45.070820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:45.070820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:45.070820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:45.070820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:45.070820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:00:45.072602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:00:45.072602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:45.072602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:45.072602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 240/474] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:45.076696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36acf8e9-d244-4957-972a-a51e62966666 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:45.211058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 235/475] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:45.272413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36acf8e9-d244-4957-972a-a51e62966666 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:45.313216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40aa2d5-2780-428c-9601-3e625ddc2cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:45.313851 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40aa2d5-2780-428c-9601-3e625ddc2cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:45.314127 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40aa2d5-2780-428c-9601-3e625ddc2cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:45.314399 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40aa2d5-2780-428c-9601-3e625ddc2cd0 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:45.315111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40aa2d5-2780-428c-9601-3e625ddc2cd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:45.315413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40aa2d5-2780-428c-9601-3e625ddc2cd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:45.316851 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40aa2d5-2780-428c-9601-3e625ddc2cd0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:45.331351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 241/476] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:45.338428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf3184-35c2-43b7-9acd-e77bee92b7b0 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:45.421499 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:45.586737 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf3184-35c2-43b7-9acd-e77bee92b7b0 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:45.649459 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ce5bf-af22-43dd-8b15-320e7a0faa4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:45.650754 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ce5bf-af22-43dd-8b15-320e7a0faa4c None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:45.651464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ce5bf-af22-43dd-8b15-320e7a0faa4c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:45.652116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ce5bf-af22-43dd-8b15-320e7a0faa4c None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:45.653383 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ce5bf-af22-43dd-8b15-320e7a0faa4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:45.655918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ce5bf-af22-43dd-8b15-320e7a0faa4c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:45.657685 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40aa2d5-2780-428c-9601-3e625ddc2cd0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:45.685017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 236/477] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 6 16:00:45 2022] POST /identity/v3/users => generated 249 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:00:45.833382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:45.937210 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40aa2d5-2780-428c-9601-3e625ddc2cd0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:46.017537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25880b3-c07e-4cd4-a420-6854f2d40d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:46.018941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25880b3-c07e-4cd4-a420-6854f2d40d8a None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:46.019656 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25880b3-c07e-4cd4-a420-6854f2d40d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:46.020328 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25880b3-c07e-4cd4-a420-6854f2d40d8a None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:46.022509 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25880b3-c07e-4cd4-a420-6854f2d40d8a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:46.025588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25880b3-c07e-4cd4-a420-6854f2d40d8a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:46.039772 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.sql.core [None req-d25880b3-c07e-4cd4-a420-6854f2d40d8a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 16:00:46.039772 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 06 16:00:46.039772 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [parameters: {'id': '58ff7adcdcb94d22bea082f144e392a6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 06 16:00:46.039772 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81686) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 16:00:46.074349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-d25880b3-c07e-4cd4-a420-6854f2d40d8a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 06 16:00:46.074349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:46.074349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 16:00:46.074349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 16:00:46.074349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 16:00:46.074349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 16:00:46.074349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 16:00:46.074349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 16:00:46.074349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 16:00:46.074349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application conn.query(q) Sep 06 16:00:46.074349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 16:00:46.074349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 16:00:46.074349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 16:00:46.074349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application result.read() Sep 06 16:00:46.074349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 16:00:46.076067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 16:00:46.076067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 16:00:46.076067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 16:00:46.076067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 16:00:46.076067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 16:00:46.076067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 16:00:46.076067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 16:00:46.076067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 16:00:46.076067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:46.076067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 16:00:46.076067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:46.076067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:46.076067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 16:00:46.076067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 16:00:46.076067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 06 16:00:46.076067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return ref.to_dict() Sep 06 16:00:46.076067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 16:00:46.076067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application next(self.gen) Sep 06 16:00:46.077793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 16:00:46.077793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application yield resource Sep 06 16:00:46.077793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 16:00:46.077793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application next(self.gen) Sep 06 16:00:46.077793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 16:00:46.077793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 16:00:46.077793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 16:00:46.077793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 16:00:46.077793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 16:00:46.077793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise self.value Sep 06 16:00:46.077793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 16:00:46.077793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 16:00:46.077793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 16:00:46.077793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application session.commit() Sep 06 16:00:46.077793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 16:00:46.077793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 16:00:46.077793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 16:00:46.079979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 16:00:46.079979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 16:00:46.079979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self.session.flush() Sep 06 16:00:46.079979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 16:00:46.079979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 16:00:46.079979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 16:00:46.079979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 16:00:46.079979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 16:00:46.079979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application compat.raise_( Sep 06 16:00:46.079979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 16:00:46.079979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception Sep 06 16:00:46.079979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 16:00:46.079979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 16:00:46.079979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 16:00:46.079979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 16:00:46.079979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 16:00:46.079979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 16:00:46.081664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 16:00:46.081664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 16:00:46.081664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 06 16:00:46.081664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 06 16:00:46.081664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 16:00:46.081664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 16:00:46.081664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 16:00:46.081664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 16:00:46.081664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 16:00:46.081664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 16:00:46.081664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 16:00:46.081664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 16:00:46.081664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 16:00:46.081664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 16:00:46.081664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 16:00:46.081664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception Sep 06 16:00:46.083288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 16:00:46.083288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 16:00:46.083288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 16:00:46.083288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 16:00:46.083288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 16:00:46.083288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 16:00:46.083288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 16:00:46.083288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application conn.query(q) Sep 06 16:00:46.083288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 16:00:46.083288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 16:00:46.083288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 16:00:46.083288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application result.read() Sep 06 16:00:46.083288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 16:00:46.083288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 16:00:46.083288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 16:00:46.083288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 16:00:46.085099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 16:00:46.085099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 16:00:46.085099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 16:00:46.085099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 16:00:46.085099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 16:00:46.085099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 06 16:00:46.085099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [parameters: {'id': '58ff7adcdcb94d22bea082f144e392a6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 06 16:00:46.085099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 16:00:46.085099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:46.085099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:00:46.085099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:46.085099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:46.085099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:46.085099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:46.085099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:46.085099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:46.086707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:46.086707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:46.086707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:46.086707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:46.086707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:46.086707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:46.086707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 06 16:00:46.086707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 06 16:00:46.086707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:46.086707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:46.086707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 06 16:00:46.086707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 06 16:00:46.086707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 16:00:46.086707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 16:00:46.086707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 06 16:00:46.086707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:46.091783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 242/478] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Sep 6 16:00:45 2022] POST /identity/v3/roles => generated 144 bytes in 433 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:00:46.100913 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ce5bf-af22-43dd-8b15-320e7a0faa4c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:46.275333 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:46.307580 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ce5bf-af22-43dd-8b15-320e7a0faa4c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:46.355397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844f9bc-d0f3-4b98-b192-6484bbfbec48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:46.356432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844f9bc-d0f3-4b98-b192-6484bbfbec48 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:46.356895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844f9bc-d0f3-4b98-b192-6484bbfbec48 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:46.357340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844f9bc-d0f3-4b98-b192-6484bbfbec48 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:46.371803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844f9bc-d0f3-4b98-b192-6484bbfbec48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:46.377195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-8844f9bc-d0f3-4b98-b192-6484bbfbec48 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:00:46.377195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:46.377195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:46.377195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:46.377195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:46.377195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:46.377195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:46.377195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:46.377195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:46.377195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:46.377195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:46.377195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:46.377195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:46.377195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:46.377195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:46.377195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:46.378136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:46.378136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:46.378136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:46.378136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:46.378136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:46.378136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:46.378136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:46.378136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:46.378136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:46.378136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:46.378136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:46.378136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:46.378136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:46.378136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:46.378136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:46.378136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:46.378954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:46.378954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:46.378954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:46.378954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:46.378954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:46.378954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:46.378954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:46.378954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:46.378954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:46.378954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:46.378954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 16:00:46.378954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:46.381963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 237/479] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 16:00:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:46.390800 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25880b3-c07e-4cd4-a420-6854f2d40d8a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:46.504333 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25880b3-c07e-4cd4-a420-6854f2d40d8a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:46.527043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:46.546418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b88afc-3f8e-4654-8643-e052470dd9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:46.547086 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b88afc-3f8e-4654-8643-e052470dd9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:46.547362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b88afc-3f8e-4654-8643-e052470dd9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:46.547632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b88afc-3f8e-4654-8643-e052470dd9b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:46.548348 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b88afc-3f8e-4654-8643-e052470dd9b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:46.548646 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b88afc-3f8e-4654-8643-e052470dd9b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:46.550297 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b88afc-3f8e-4654-8643-e052470dd9b8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:46.560672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 243/480] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:00:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:47.143683 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd3fea-c56b-4441-b0ee-54f62efd2fc9 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:47.144207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd3fea-c56b-4441-b0ee-54f62efd2fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:47.144672 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd3fea-c56b-4441-b0ee-54f62efd2fc9 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:47.147522 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 238/481] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:47.156851 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfe619-654b-4f02-b1c5-be00305fd9e9 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:47.157381 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfe619-654b-4f02-b1c5-be00305fd9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:47.157834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfe619-654b-4f02-b1c5-be00305fd9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:47.191958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-2edfe619-654b-4f02-b1c5-be00305fd9e9 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:47.194586 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2edfe619-654b-4f02-b1c5-be00305fd9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:47.251379 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1303cb7-5c48-412e-856a-f03c2acb3997 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:47.252052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1303cb7-5c48-412e-856a-f03c2acb3997 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:47.252527 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1303cb7-5c48-412e-856a-f03c2acb3997 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:47.254289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 239/482] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:47.261100 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335ed68-e164-467b-a282-e826bc4e4ac7 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:47.261717 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335ed68-e164-467b-a282-e826bc4e4ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:47.262203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335ed68-e164-467b-a282-e826bc4e4ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:47.289098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4335ed68-e164-467b-a282-e826bc4e4ac7 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:47.293165 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4335ed68-e164-467b-a282-e826bc4e4ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:47.337551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 244/483] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:47.399310 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 240/484] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:00:47.833831 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfe619-654b-4f02-b1c5-be00305fd9e9 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:48.036874 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfe619-654b-4f02-b1c5-be00305fd9e9 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:48.047798 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335ed68-e164-467b-a282-e826bc4e4ac7 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:48.071237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:48.153478 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22bf16-2068-41b3-ad63-8d4160737149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:48.154391 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22bf16-2068-41b3-ad63-8d4160737149 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:48.154870 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22bf16-2068-41b3-ad63-8d4160737149 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:48.155315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22bf16-2068-41b3-ad63-8d4160737149 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:48.189872 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:48.201100 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22bf16-2068-41b3-ad63-8d4160737149 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:48.218480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-fe22bf16-2068-41b3-ad63-8d4160737149 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:00:48.218480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:48.218480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:48.218480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:48.218480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:48.218480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:48.218480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:48.218480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:48.218480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:48.218480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:48.218480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:48.218480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:48.218480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:48.218480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:48.218480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:48.218480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:48.219558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:48.219558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:48.219558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:48.219558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:48.219558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:48.219558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:48.219558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:48.219558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:48.219558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:48.219558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:48.219558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:48.219558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:48.219558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:48.219558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:48.219558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:48.219558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:48.220354 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:48.220354 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:48.220354 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:48.220354 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:48.220354 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:48.220354 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:48.220354 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:48.220354 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:48.220354 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:48.220354 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:48.220354 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:00:48.220354 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:48.223393 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 245/485] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:00:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:48.236226 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22bf16-2068-41b3-ad63-8d4160737149 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:48.291187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335ed68-e164-467b-a282-e826bc4e4ac7 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:48.337284 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70febc79-bf28-4511-ae50-654cd647c7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:48.338421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70febc79-bf28-4511-ae50-654cd647c7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:48.338927 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70febc79-bf28-4511-ae50-654cd647c7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:48.339413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70febc79-bf28-4511-ae50-654cd647c7a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:48.352212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70febc79-bf28-4511-ae50-654cd647c7a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:48.357841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-70febc79-bf28-4511-ae50-654cd647c7a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:00:48.357841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:48.357841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:48.357841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:48.357841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:48.357841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:48.357841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:48.357841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:48.357841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:48.357841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:48.357841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:48.357841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:48.357841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:48.357841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:48.357841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:48.357841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:48.358813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:48.358813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:48.358813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:48.358813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:48.358813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:48.358813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:48.358813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:48.358813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:48.358813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:48.358813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:48.358813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:48.358813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:48.358813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:48.358813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:48.358813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:48.358813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:48.359616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:48.359616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:48.359616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:48.359616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:48.359616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:48.359616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:48.359616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:48.359616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:48.359616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:48.359616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:48.359616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:00:48.359616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:48.366698 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 241/486] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:00:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:48.368641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08ffc8-a05e-4530-be59-9f2291e41372 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:48.369143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08ffc8-a05e-4530-be59-9f2291e41372 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:48.369666 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08ffc8-a05e-4530-be59-9f2291e41372 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:48.371587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 242/487] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:48.375913 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08ffc8-a05e-4530-be59-9f2291e41372 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:48.452492 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22bf16-2068-41b3-ad63-8d4160737149 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:48.513057 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08ffc8-a05e-4530-be59-9f2291e41372 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:48.514753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45438d-36d3-447f-b5f9-a527fcbffc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:48.516185 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45438d-36d3-447f-b5f9-a527fcbffc12 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:48.518090 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45438d-36d3-447f-b5f9-a527fcbffc12 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:48.518561 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45438d-36d3-447f-b5f9-a527fcbffc12 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:48.520008 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45438d-36d3-447f-b5f9-a527fcbffc12 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:48.520614 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45438d-36d3-447f-b5f9-a527fcbffc12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:48.523442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45438d-36d3-447f-b5f9-a527fcbffc12 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:48.548412 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 246/488] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:00:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:48.551251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:48.551465 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23336d93-3ba6-4f04-a2d6-48aa99cea3f6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:48.551764 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23336d93-3ba6-4f04-a2d6-48aa99cea3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:48.551946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23336d93-3ba6-4f04-a2d6-48aa99cea3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:48.576344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c59b69-5fed-42a1-99ca-90e38bd69cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:48.577872 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c59b69-5fed-42a1-99ca-90e38bd69cfa None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:48.578677 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c59b69-5fed-42a1-99ca-90e38bd69cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:48.579451 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c59b69-5fed-42a1-99ca-90e38bd69cfa None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:48.580071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-23336d93-3ba6-4f04-a2d6-48aa99cea3f6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:48.581288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-23336d93-3ba6-4f04-a2d6-48aa99cea3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:48.581667 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c59b69-5fed-42a1-99ca-90e38bd69cfa None admin] RBAC: Adding query filter params (name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:48.582488 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c59b69-5fed-42a1-99ca-90e38bd69cfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:48.587506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c59b69-5fed-42a1-99ca-90e38bd69cfa None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:48.612070 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 243/489] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:00:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:48.614090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c59b69-5fed-42a1-99ca-90e38bd69cfa None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:48.630363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:48.687106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 247/490] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:48.688192 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23336d93-3ba6-4f04-a2d6-48aa99cea3f6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:48.898645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c59b69-5fed-42a1-99ca-90e38bd69cfa None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:48.900784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23336d93-3ba6-4f04-a2d6-48aa99cea3f6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:48.956936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3cd83-24a0-4a92-817c-7c2ab149d932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:48.957837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3cd83-24a0-4a92-817c-7c2ab149d932 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:48.958259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3cd83-24a0-4a92-817c-7c2ab149d932 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:48.959465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3cd83-24a0-4a92-817c-7c2ab149d932 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:48.977718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3cd83-24a0-4a92-817c-7c2ab149d932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:48.986026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3cd83-24a0-4a92-817c-7c2ab149d932 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:48.986196 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb09695-aae1-4d58-8bd0-975e5704e691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:48.986196 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb09695-aae1-4d58-8bd0-975e5704e691 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:48.986196 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb09695-aae1-4d58-8bd0-975e5704e691 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:48.986196 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb09695-aae1-4d58-8bd0-975e5704e691 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:49.004001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb09695-aae1-4d58-8bd0-975e5704e691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:49.007514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb09695-aae1-4d58-8bd0-975e5704e691 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:49.012661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-c2b3cd83-24a0-4a92-817c-7c2ab149d932 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:49.012661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:49.012661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:49.012661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:49.012661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:49.012661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:49.012661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:49.012661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:49.012661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:49.012661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:49.012661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:49.012661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:49.012661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:00:49.012661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:00:49.012661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:00:49.012661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:00:49.014428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:49.014428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:49.014428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:49.014428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:49.014428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:49.014428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:49.014428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:49.014428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:49.014428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:49.014428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:49.014428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:49.014428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:49.014428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:49.014428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:49.014428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:49.014428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:49.014428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:00:49.016876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:00:49.016876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:49.016876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:49.023232 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 244/491] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:49.034606 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3cd83-24a0-4a92-817c-7c2ab149d932 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:49.064531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-9bb09695-aae1-4d58-8bd0-975e5704e691 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:49.064531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:49.064531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:49.064531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:49.064531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:49.064531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:49.064531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:49.064531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:49.064531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:49.064531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:49.064531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:49.064531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:49.064531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:00:49.064531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:00:49.064531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:00:49.064531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:00:49.066289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:49.066289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:49.066289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:49.066289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:49.066289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:49.066289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:49.066289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:49.066289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:49.066289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:49.066289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:49.066289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:49.066289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:49.066289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:49.066289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:49.066289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:49.066289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:49.066289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:00:49.068662 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:00:49.068662 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:49.068662 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:49.075469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 248/492] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:49.082581 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb09695-aae1-4d58-8bd0-975e5704e691 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:49.113554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:49.167233 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:49.274893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb09695-aae1-4d58-8bd0-975e5704e691 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:49.275678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3cd83-24a0-4a92-817c-7c2ab149d932 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:49.334430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d483d04-4f39-417c-a578-089e3496b4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:49.335460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d483d04-4f39-417c-a578-089e3496b4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:49.335946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d483d04-4f39-417c-a578-089e3496b4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:49.336414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d483d04-4f39-417c-a578-089e3496b4c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:49.337494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d483d04-4f39-417c-a578-089e3496b4c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:49.337883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96cc04-3c4f-4971-96c7-67e061e2d3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:49.338058 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d483d04-4f39-417c-a578-089e3496b4c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:49.339290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96cc04-3c4f-4971-96c7-67e061e2d3ae None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:49.340019 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96cc04-3c4f-4971-96c7-67e061e2d3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:49.340521 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d483d04-4f39-417c-a578-089e3496b4c1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:49.341015 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96cc04-3c4f-4971-96c7-67e061e2d3ae None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:49.342372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96cc04-3c4f-4971-96c7-67e061e2d3ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:49.343174 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96cc04-3c4f-4971-96c7-67e061e2d3ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:49.345916 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96cc04-3c4f-4971-96c7-67e061e2d3ae None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:49.362463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 249/493] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:49.364365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a711fd-fc89-4210-a84d-cfc43349d53d None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:49.364779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a711fd-fc89-4210-a84d-cfc43349d53d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:49.365249 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a711fd-fc89-4210-a84d-cfc43349d53d None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:49.366978 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 250/494] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:49.369531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a711fd-fc89-4210-a84d-cfc43349d53d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:49.374268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 245/495] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:49.375854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96cc04-3c4f-4971-96c7-67e061e2d3ae None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:49.482957 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:49.507117 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96cc04-3c4f-4971-96c7-67e061e2d3ae None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:49.508811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:49.523041 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a711fd-fc89-4210-a84d-cfc43349d53d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:49.546612 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0258f9-c8c1-431c-be56-a599be888a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:49.547367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0258f9-c8c1-431c-be56-a599be888a4d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:49.547648 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0258f9-c8c1-431c-be56-a599be888a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:49.547921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0258f9-c8c1-431c-be56-a599be888a4d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:49.557289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0258f9-c8c1-431c-be56-a599be888a4d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:49.558780 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0258f9-c8c1-431c-be56-a599be888a4d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:49.573506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-df0258f9-c8c1-431c-be56-a599be888a4d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 16:00:49.573506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:49.573506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:49.573506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:49.573506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:49.573506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:49.573506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:49.573506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:49.573506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:49.573506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:49.573506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:49.573506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:49.573506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:00:49.573506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:00:49.573506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:00:49.573506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:00:49.574494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:49.574494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:49.574494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:00:49.574494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:00:49.574494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:00:49.574494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:00:49.574494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:49.574494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:49.574494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:49.574494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:49.574494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:49.574494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:49.574494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:49.574494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:49.574494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:49.574494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:49.574494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:49.574494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:49.580729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:49.580729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:49.580729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:00:49.580729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:00:49.580729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:00:49.580729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:00:49.580729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:00:49.580729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:00:49.580729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 16:00:49.580729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:49.580729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 246/496] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:49.580729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fb279-02cf-4821-abff-4f2a3f3f2fb5 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:49.580729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fb279-02cf-4821-abff-4f2a3f3f2fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:49.581421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fb279-02cf-4821-abff-4f2a3f3f2fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:49.581485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1ad0f-6003-4547-bcec-bc9e7568251d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:49.581485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1ad0f-6003-4547-bcec-bc9e7568251d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:49.582037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1ad0f-6003-4547-bcec-bc9e7568251d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:49.582205 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1ad0f-6003-4547-bcec-bc9e7568251d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:49.596814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-088fb279-02cf-4821-abff-4f2a3f3f2fb5 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:49.598047 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-088fb279-02cf-4821-abff-4f2a3f3f2fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:49.607901 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d1ad0f-6003-4547-bcec-bc9e7568251d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:49.613180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-c1d1ad0f-6003-4547-bcec-bc9e7568251d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:00:49.613180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:49.613180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:49.613180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:49.613180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:49.613180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:49.613180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:49.613180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:49.613180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:49.613180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:49.613180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:49.613180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:49.613180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:49.613180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:49.613180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:49.613180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:49.615456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:49.615456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:49.615456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:49.615456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:49.615456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:49.615456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:49.615456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:49.615456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:49.615456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:49.615456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:49.615456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:49.615456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:49.615456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:49.615456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:49.615456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:49.615456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:49.617848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:49.617848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:49.617848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:49.617848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:49.617848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:49.617848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:49.617848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:49.617848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:49.617848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:49.617848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:49.617848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:00:49.617848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:49.617848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 251/497] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:00:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:49.617848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1ad0f-6003-4547-bcec-bc9e7568251d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:49.706695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 247/498] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:49.706695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088fb279-02cf-4821-abff-4f2a3f3f2fb5 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:49.790736 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1ad0f-6003-4547-bcec-bc9e7568251d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:49.837299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13165d-c71c-4afd-9005-8a000b762c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:49.838073 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13165d-c71c-4afd-9005-8a000b762c56 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:49.838422 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13165d-c71c-4afd-9005-8a000b762c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:49.838808 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13165d-c71c-4afd-9005-8a000b762c56 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:49.841224 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:49.849101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13165d-c71c-4afd-9005-8a000b762c56 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:49.851265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13165d-c71c-4afd-9005-8a000b762c56 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:49.871525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-7f13165d-c71c-4afd-9005-8a000b762c56 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:00:49.871525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:49.871525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:49.871525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:49.871525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:49.871525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:49.871525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:49.871525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:49.871525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:49.871525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:49.871525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:49.871525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:49.871525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:00:49.871525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:00:49.871525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:00:49.871525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:00:49.872541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:49.872541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:49.872541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:00:49.872541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:00:49.872541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:00:49.872541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:00:49.872541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:49.872541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:49.872541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:49.872541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:49.872541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:49.872541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:49.872541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:49.872541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:49.872541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:49.872541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:49.872541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:49.872541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:49.873435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:49.873435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:49.873435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:00:49.873435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:00:49.873435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:00:49.873435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:00:49.873435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:00:49.873435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:00:49.873435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:00:49.873435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:49.873435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 252/499] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Sep 6 16:00:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:49.873989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13165d-c71c-4afd-9005-8a000b762c56 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:49.900396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:49.917624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088fb279-02cf-4821-abff-4f2a3f3f2fb5 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:49.980093 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7eb9e-b13b-41db-a60a-4726be21ca1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:49.980973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7eb9e-b13b-41db-a60a-4726be21ca1e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:49.981469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7eb9e-b13b-41db-a60a-4726be21ca1e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:49.981962 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7eb9e-b13b-41db-a60a-4726be21ca1e None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:49.983368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7eb9e-b13b-41db-a60a-4726be21ca1e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:49.983873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7eb9e-b13b-41db-a60a-4726be21ca1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:49.986425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7eb9e-b13b-41db-a60a-4726be21ca1e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:49.993767 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13165d-c71c-4afd-9005-8a000b762c56 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:50.031830 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35d762-d8ef-4923-aca9-4658d08996f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:50.032442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35d762-d8ef-4923-aca9-4658d08996f8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:50.032717 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35d762-d8ef-4923-aca9-4658d08996f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:50.033582 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35d762-d8ef-4923-aca9-4658d08996f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:50.034256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35d762-d8ef-4923-aca9-4658d08996f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:50.034574 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35d762-d8ef-4923-aca9-4658d08996f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:50.035995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35d762-d8ef-4923-aca9-4658d08996f8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:50.036726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 248/500] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Sep 6 16:00:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:50.038093 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7eb9e-b13b-41db-a60a-4726be21ca1e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:50.046455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 253/501] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:00:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:50.047937 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fcd02-5c97-4c12-94af-3cd11fefd686 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:50.048391 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fcd02-5c97-4c12-94af-3cd11fefd686 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:50.048917 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fcd02-5c97-4c12-94af-3cd11fefd686 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:50.050713 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 254/502] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:50.051295 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fcd02-5c97-4c12-94af-3cd11fefd686 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:50.116189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:50.162837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7eb9e-b13b-41db-a60a-4726be21ca1e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:50.165308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fcd02-5c97-4c12-94af-3cd11fefd686 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:50.200024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9b67f-b506-4aef-9d8c-a3639de60afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:50.200631 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9b67f-b506-4aef-9d8c-a3639de60afc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:50.200902 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9b67f-b506-4aef-9d8c-a3639de60afc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:50.201164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9b67f-b506-4aef-9d8c-a3639de60afc None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:50.201890 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f9b67f-b506-4aef-9d8c-a3639de60afc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:50.202189 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f9b67f-b506-4aef-9d8c-a3639de60afc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:50.203654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f9b67f-b506-4aef-9d8c-a3639de60afc None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:50.214116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c345ceb-9249-470e-8800-543c4f558ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:50.215251 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c345ceb-9249-470e-8800-543c4f558ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:50.217012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c345ceb-9249-470e-8800-543c4f558ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:50.217507 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c345ceb-9249-470e-8800-543c4f558ac5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:50.223666 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 249/503] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Sep 6 16:00:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:50.225054 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9b67f-b506-4aef-9d8c-a3639de60afc None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:50.235583 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c345ceb-9249-470e-8800-543c4f558ac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:50.236399 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:50.242323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-8c345ceb-9249-470e-8800-543c4f558ac5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:00:50.242323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:50.242323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:50.242323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:50.242323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:50.242323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:50.242323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:50.242323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:50.242323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:50.242323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:50.242323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:50.242323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:50.242323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:50.242323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:50.242323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:50.242323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:50.243406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:50.243406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:50.243406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:50.243406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:50.243406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:50.243406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:50.243406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:50.243406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:50.243406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:50.243406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:50.243406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:50.243406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:50.243406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:50.243406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:50.243406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:50.243406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:50.244692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:50.244692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:50.244692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:50.244692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:50.244692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:50.244692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:50.244692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:50.244692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:50.244692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:50.244692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:50.244692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:00:50.244692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:50.244692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 255/504] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:00:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:50.244692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c345ceb-9249-470e-8800-543c4f558ac5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:50.360824 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9b67f-b506-4aef-9d8c-a3639de60afc None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:50.371453 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c345ceb-9249-470e-8800-543c4f558ac5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:50.390361 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:50.411128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fbd9c-c5c5-4adc-8685-b0c48eb65210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:50.412624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fbd9c-c5c5-4adc-8685-b0c48eb65210 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:50.413020 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fbd9c-c5c5-4adc-8685-b0c48eb65210 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:50.413376 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fbd9c-c5c5-4adc-8685-b0c48eb65210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:50.421720 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073347f-c581-4fde-ba9b-28e8330f0d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:50.422814 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073347f-c581-4fde-ba9b-28e8330f0d39 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:50.423302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073347f-c581-4fde-ba9b-28e8330f0d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:50.423768 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073347f-c581-4fde-ba9b-28e8330f0d39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:50.443618 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073347f-c581-4fde-ba9b-28e8330f0d39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:50.446368 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073347f-c581-4fde-ba9b-28e8330f0d39 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:50.472834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-f073347f-c581-4fde-ba9b-28e8330f0d39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:50.472834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:50.472834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:50.472834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:50.472834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:50.472834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:50.472834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:50.472834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:50.472834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:50.472834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:50.472834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:50.472834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:50.472834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:00:50.472834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:00:50.472834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:00:50.472834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:00:50.474709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:50.474709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:50.474709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:50.474709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:50.474709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:50.474709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:50.474709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:50.474709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:50.474709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:50.474709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:50.474709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:50.474709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:50.474709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:50.474709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:50.474709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:50.474709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:50.474709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:00:50.476519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:00:50.476519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:50.476519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:50.476519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 256/505] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:50.477021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d4b3e-b577-4e8f-9913-872250559fe3 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:50.477197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d4b3e-b577-4e8f-9913-872250559fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:50.477676 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d4b3e-b577-4e8f-9913-872250559fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:50.496522 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fbd9c-c5c5-4adc-8685-b0c48eb65210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:50.497988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fbd9c-c5c5-4adc-8685-b0c48eb65210 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:50.505609 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-4a7d4b3e-b577-4e8f-9913-872250559fe3 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:50.507429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4a7d4b3e-b577-4e8f-9913-872250559fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:50.509334 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:50.622418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 257/506] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:50.623256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d4b3e-b577-4e8f-9913-872250559fe3 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:50.638839 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 250/507] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:00:50.641107 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fbd9c-c5c5-4adc-8685-b0c48eb65210 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:50.765605 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:50.771319 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d4b3e-b577-4e8f-9913-872250559fe3 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:50.800407 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fbd9c-c5c5-4adc-8685-b0c48eb65210 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:50.813973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:50.822701 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8999c-c296-4029-966e-ce70b5e497fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:50.823927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8999c-c296-4029-966e-ce70b5e497fe None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:50.823927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8999c-c296-4029-966e-ce70b5e497fe None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:50.823927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8999c-c296-4029-966e-ce70b5e497fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:50.870952 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c6b1b-014b-4753-ba68-687a9033412c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:50.872493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c6b1b-014b-4753-ba68-687a9033412c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:50.873156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c6b1b-014b-4753-ba68-687a9033412c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:50.873806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c6b1b-014b-4753-ba68-687a9033412c None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:50.875291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c6b1b-014b-4753-ba68-687a9033412c None admin] RBAC: Adding query filter params (name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:50.876258 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c6b1b-014b-4753-ba68-687a9033412c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:50.879971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c6b1b-014b-4753-ba68-687a9033412c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:50.895299 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 251/508] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:00:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:50.896437 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c6b1b-014b-4753-ba68-687a9033412c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:50.905328 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8999c-c296-4029-966e-ce70b5e497fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:50.908033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8999c-c296-4029-966e-ce70b5e497fe None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:51.016168 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:51.017979 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c6b1b-014b-4753-ba68-687a9033412c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:51.028385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 258/509] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:00:51.028385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8999c-c296-4029-966e-ce70b5e497fe None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:51.057497 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65574ee9-5840-4a73-84a2-bb8f8f7db9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:51.058105 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65574ee9-5840-4a73-84a2-bb8f8f7db9bb None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:51.058386 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65574ee9-5840-4a73-84a2-bb8f8f7db9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:51.058654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65574ee9-5840-4a73-84a2-bb8f8f7db9bb None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:51.059332 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65574ee9-5840-4a73-84a2-bb8f8f7db9bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:51.059632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65574ee9-5840-4a73-84a2-bb8f8f7db9bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:51.061023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65574ee9-5840-4a73-84a2-bb8f8f7db9bb None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:51.073357 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 252/510] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:51.073357 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65574ee9-5840-4a73-84a2-bb8f8f7db9bb None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:51.107469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:51.141765 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8999c-c296-4029-966e-ce70b5e497fe None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:51.187068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26c28a-3341-4594-a157-44f441c82dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:51.187708 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26c28a-3341-4594-a157-44f441c82dfc None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:51.187981 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26c28a-3341-4594-a157-44f441c82dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:51.188250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26c28a-3341-4594-a157-44f441c82dfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:51.194199 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65574ee9-5840-4a73-84a2-bb8f8f7db9bb None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:51.200645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26c28a-3341-4594-a157-44f441c82dfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:51.203237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26c28a-3341-4594-a157-44f441c82dfc None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:51.219246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-9b26c28a-3341-4594-a157-44f441c82dfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:51.219246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:51.219246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:51.219246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:51.219246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:51.219246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:51.219246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:51.219246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:51.219246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:51.219246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:51.219246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:51.219246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:51.219246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:00:51.219246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:00:51.219246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:00:51.219246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:00:51.220380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:51.220380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:51.220380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:51.220380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:51.220380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:51.220380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:51.220380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:51.220380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:51.220380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:51.220380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:51.220380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:51.220380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:51.220380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:51.220380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:51.220380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:51.220380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:51.220380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:00:51.221299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:00:51.221299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:51.221299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:51.221299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 259/511] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:51.221299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26c28a-3341-4594-a157-44f441c82dfc None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:51.244379 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65aeafd-91c2-4a39-b58d-1246058d3803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:51.245019 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aeafd-91c2-4a39-b58d-1246058d3803 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:51.245291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aeafd-91c2-4a39-b58d-1246058d3803 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:51.245557 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aeafd-91c2-4a39-b58d-1246058d3803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:51.257593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65aeafd-91c2-4a39-b58d-1246058d3803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:51.259118 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65aeafd-91c2-4a39-b58d-1246058d3803 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:51.275484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-e65aeafd-91c2-4a39-b58d-1246058d3803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:51.275484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:51.275484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:51.275484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:51.275484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:51.275484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:51.275484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:51.275484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:51.275484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:51.275484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:51.275484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:51.275484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:51.275484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:00:51.275484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:00:51.275484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:00:51.275484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:00:51.279139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:51.279139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:51.279139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:51.279139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:51.279139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:51.279139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:51.279139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:51.279139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:51.279139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:51.279139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:51.279139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:51.279139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:51.279139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:51.279139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:51.279139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:51.279139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:51.279139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:00:51.279987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:00:51.279987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:51.279987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:51.279987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 253/512] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:51.279987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65aeafd-91c2-4a39-b58d-1246058d3803 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:51.284918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:51.356368 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26c28a-3341-4594-a157-44f441c82dfc None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:51.378292 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:51.394620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac9a3c-73a3-4815-b967-b217f977d063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:51.395874 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac9a3c-73a3-4815-b967-b217f977d063 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:51.396822 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac9a3c-73a3-4815-b967-b217f977d063 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:51.397247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac9a3c-73a3-4815-b967-b217f977d063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:51.398915 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65aeafd-91c2-4a39-b58d-1246058d3803 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:51.410144 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac9a3c-73a3-4815-b967-b217f977d063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:51.411763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac9a3c-73a3-4815-b967-b217f977d063 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:51.427367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-0fac9a3c-73a3-4815-b967-b217f977d063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:51.427367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:51.427367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:51.427367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:51.427367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:51.427367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:51.427367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:51.427367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:51.427367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:51.427367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:51.427367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:51.427367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:51.427367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:00:51.427367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:00:51.427367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:00:51.427367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:00:51.429296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:51.429296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:51.429296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:51.429296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:51.429296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:51.429296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:51.429296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:51.429296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:51.429296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:51.429296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:51.429296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:51.429296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:51.429296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:51.429296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:51.429296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:51.429296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:51.429296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:00:51.431030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:00:51.431030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:51.431030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:51.431030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 260/513] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:51.431030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac9a3c-73a3-4815-b967-b217f977d063 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:51.441364 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b62e9b-217f-4c06-9634-49a961e0435d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:51.441580 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b62e9b-217f-4c06-9634-49a961e0435d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:51.441744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b62e9b-217f-4c06-9634-49a961e0435d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:51.441958 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b62e9b-217f-4c06-9634-49a961e0435d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:51.455517 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b62e9b-217f-4c06-9634-49a961e0435d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:51.456975 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b62e9b-217f-4c06-9634-49a961e0435d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:51.472340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-80b62e9b-217f-4c06-9634-49a961e0435d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:51.472340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:51.472340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:51.472340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:51.472340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:51.472340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:51.472340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:51.472340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:51.472340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:51.472340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:51.472340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:51.472340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:51.472340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:00:51.472340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:00:51.472340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:00:51.472340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:00:51.473537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:51.473537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:51.473537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:51.473537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:51.473537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:51.473537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:51.473537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:51.473537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:51.473537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:51.473537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:51.473537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:51.473537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:51.473537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:51.473537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:51.473537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:51.473537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:51.473537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:00:51.474507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:00:51.474507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:51.474507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:51.474507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 254/514] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:51.474507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b62e9b-217f-4c06-9634-49a961e0435d None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:51.561455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac9a3c-73a3-4815-b967-b217f977d063 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:51.563271 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:51.599038 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b62e9b-217f-4c06-9634-49a961e0435d None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:51.607428 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a645c9-8580-4530-9a68-4912488f0da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:51.608096 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a645c9-8580-4530-9a68-4912488f0da3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:51.608369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a645c9-8580-4530-9a68-4912488f0da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:51.608641 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a645c9-8580-4530-9a68-4912488f0da3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:51.617441 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a645c9-8580-4530-9a68-4912488f0da3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:51.618958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a645c9-8580-4530-9a68-4912488f0da3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:51.638235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-10a645c9-8580-4530-9a68-4912488f0da3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 16:00:51.638235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:51.638235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:51.638235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:51.638235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:51.638235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:51.638235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:51.638235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:51.638235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:51.638235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:51.638235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:51.638235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:51.638235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:00:51.638235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:00:51.638235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:00:51.638235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:00:51.640106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:51.640106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:51.640106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:00:51.640106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:00:51.640106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:00:51.640106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:00:51.640106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:51.640106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:51.640106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:51.640106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:51.640106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:51.640106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:51.640106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:51.640106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:51.640106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:51.640106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:51.640106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:51.640106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:51.641427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:51.641427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:51.641427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:00:51.641427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:00:51.641427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:00:51.641427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:00:51.641427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:00:51.641427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:00:51.641427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 16:00:51.641427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:51.641928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92f9d0-1cce-4c30-b877-1dd618b760a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:51.641928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92f9d0-1cce-4c30-b877-1dd618b760a8 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:51.642098 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 261/515] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:51.642098 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a645c9-8580-4530-9a68-4912488f0da3 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:51.642474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92f9d0-1cce-4c30-b877-1dd618b760a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:51.643501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92f9d0-1cce-4c30-b877-1dd618b760a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:51.645454 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92f9d0-1cce-4c30-b877-1dd618b760a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:51.645715 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92f9d0-1cce-4c30-b877-1dd618b760a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:51.647728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92f9d0-1cce-4c30-b877-1dd618b760a8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:51.649567 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:51.661214 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 255/516] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:51.662125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92f9d0-1cce-4c30-b877-1dd618b760a8 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:51.758431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a645c9-8580-4530-9a68-4912488f0da3 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:51.789469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92f9d0-1cce-4c30-b877-1dd618b760a8 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:51.801911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3ab26-ba92-4fe7-b632-7815ff8cf8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:51.803351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3ab26-ba92-4fe7-b632-7815ff8cf8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:51.804325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3ab26-ba92-4fe7-b632-7815ff8cf8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:51.804325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3ab26-ba92-4fe7-b632-7815ff8cf8c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:51.805104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda3ab26-ba92-4fe7-b632-7815ff8cf8c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:51.805413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda3ab26-ba92-4fe7-b632-7815ff8cf8c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:51.806856 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda3ab26-ba92-4fe7-b632-7815ff8cf8c3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:51.821683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 262/517] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:51.822545 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3ab26-ba92-4fe7-b632-7815ff8cf8c3 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:51.831683 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e19e92-9114-431a-b921-4802b75e156b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:51.832302 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e19e92-9114-431a-b921-4802b75e156b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:51.832575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e19e92-9114-431a-b921-4802b75e156b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:51.832842 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e19e92-9114-431a-b921-4802b75e156b None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:51.833483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e19e92-9114-431a-b921-4802b75e156b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:51.834349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e19e92-9114-431a-b921-4802b75e156b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:51.836212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e19e92-9114-431a-b921-4802b75e156b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:51.854555 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 256/518] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:51.854555 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e19e92-9114-431a-b921-4802b75e156b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:51.854873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:51.910863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:51.938340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3ab26-ba92-4fe7-b632-7815ff8cf8c3 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:51.961292 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e19e92-9114-431a-b921-4802b75e156b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:51.977325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985eafe-deb5-4411-98a6-462385e0e8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:51.977940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985eafe-deb5-4411-98a6-462385e0e8be None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:51.978215 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985eafe-deb5-4411-98a6-462385e0e8be None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:51.978481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985eafe-deb5-4411-98a6-462385e0e8be None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:51.979165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985eafe-deb5-4411-98a6-462385e0e8be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:51.979925 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985eafe-deb5-4411-98a6-462385e0e8be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:51.980856 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985eafe-deb5-4411-98a6-462385e0e8be None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:51.993485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 263/519] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:51.994337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985eafe-deb5-4411-98a6-462385e0e8be None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:51.999198 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71207d9a-efe3-40de-9054-550d5662206c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:51.999875 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71207d9a-efe3-40de-9054-550d5662206c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:52.000220 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71207d9a-efe3-40de-9054-550d5662206c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:52.000490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71207d9a-efe3-40de-9054-550d5662206c None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:52.001218 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71207d9a-efe3-40de-9054-550d5662206c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:52.001520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71207d9a-efe3-40de-9054-550d5662206c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:52.003002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71207d9a-efe3-40de-9054-550d5662206c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:52.023303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 257/520] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Sep 6 16:00:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:52.024162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71207d9a-efe3-40de-9054-550d5662206c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:52.090145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:52.115773 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985eafe-deb5-4411-98a6-462385e0e8be None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:52.163241 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71207d9a-efe3-40de-9054-550d5662206c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:52.171472 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b29373-6fe5-4cac-8831-1c74eb132e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:52.172144 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b29373-6fe5-4cac-8831-1c74eb132e4f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:52.172427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b29373-6fe5-4cac-8831-1c74eb132e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:52.172701 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b29373-6fe5-4cac-8831-1c74eb132e4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:52.181195 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b29373-6fe5-4cac-8831-1c74eb132e4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:52.182706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b29373-6fe5-4cac-8831-1c74eb132e4f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:52.187760 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:52.196906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-a4b29373-6fe5-4cac-8831-1c74eb132e4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:00:52.196906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:52.196906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:52.196906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:52.196906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:52.196906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:52.196906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:52.196906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:52.196906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:52.196906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:52.196906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:52.196906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:52.196906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:00:52.196906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:00:52.196906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:00:52.198235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:00:52.198235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:52.198235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:52.198235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:52.198235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:52.198235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:52.198235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:52.198235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:52.198235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:52.198235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:52.198235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:52.198235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:52.198235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:52.198235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:52.198235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:52.198235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:52.198235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:52.199113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:00:52.199113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:00:52.199113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:00:52.199113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:00:52.199113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:00:52.199113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:00:52.199113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:00:52.199113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:52.199113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 264/521] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:00:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:52.199113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b29373-6fe5-4cac-8831-1c74eb132e4f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:52.225378 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6498b-5177-40d6-8e3c-73686b729f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:52.226001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6498b-5177-40d6-8e3c-73686b729f68 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:52.226277 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6498b-5177-40d6-8e3c-73686b729f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:52.226548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6498b-5177-40d6-8e3c-73686b729f68 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:52.227256 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6498b-5177-40d6-8e3c-73686b729f68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:52.228753 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6498b-5177-40d6-8e3c-73686b729f68 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:52.260803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 258/522] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 6 16:00:52 2022] POST /identity/v3/users => generated 247 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:00:52.261537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6498b-5177-40d6-8e3c-73686b729f68 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:52.350291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b29373-6fe5-4cac-8831-1c74eb132e4f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:52.410222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b9d9a-0004-486a-9704-24876cd67284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:52.411348 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b9d9a-0004-486a-9704-24876cd67284 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:52.412739 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b9d9a-0004-486a-9704-24876cd67284 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:52.412858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b9d9a-0004-486a-9704-24876cd67284 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:52.428504 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b9d9a-0004-486a-9704-24876cd67284 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:52.431687 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b9d9a-0004-486a-9704-24876cd67284 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:52.448156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6498b-5177-40d6-8e3c-73686b729f68 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:52.453818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-1d0b9d9a-0004-486a-9704-24876cd67284 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 16:00:52.453818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:52.453818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:52.453818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:52.453818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:52.453818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:52.453818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:52.453818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:52.453818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:52.453818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:52.453818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:52.453818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:52.453818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:00:52.453818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:00:52.453818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:00:52.453818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:00:52.454829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:52.454829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:52.454829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:00:52.454829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:00:52.454829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:00:52.454829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:00:52.454829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:52.454829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:52.454829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:52.454829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:52.454829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:52.454829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:52.454829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:52.454829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:52.454829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:52.454829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:52.454829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:52.454829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:52.455864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:52.455864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:52.455864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:00:52.455864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:00:52.455864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:00:52.455864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:00:52.455864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:00:52.455864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:00:52.455864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 16:00:52.455864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:52.462946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 265/523] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:00:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:52.464579 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b9d9a-0004-486a-9704-24876cd67284 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:52.468971 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:52.504243 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e0159-5a92-4d28-8b74-98fcec172977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:52.505367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e0159-5a92-4d28-8b74-98fcec172977 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:52.505818 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e0159-5a92-4d28-8b74-98fcec172977 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:52.506457 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e0159-5a92-4d28-8b74-98fcec172977 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:52.516265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e0159-5a92-4d28-8b74-98fcec172977 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:52.517753 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e0159-5a92-4d28-8b74-98fcec172977 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:52.538784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-b89e0159-5a92-4d28-8b74-98fcec172977 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:00:52.538784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:52.538784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:52.538784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:52.538784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:52.538784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:52.538784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:52.538784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:52.538784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:52.538784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:52.538784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:52.538784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:52.538784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:00:52.538784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:00:52.538784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:00:52.539744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:00:52.539744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:52.539744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:52.539744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:52.539744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:52.539744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:52.539744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:52.539744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:52.539744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:52.539744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:52.539744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:52.539744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:52.539744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:52.539744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:52.539744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:52.539744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:52.539744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:52.540592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:00:52.540592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:00:52.540592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:00:52.540592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:00:52.540592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:00:52.540592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:00:52.540592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:00:52.540592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:52.540592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 259/524] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:00:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:52.541531 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e0159-5a92-4d28-8b74-98fcec172977 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:52.603536 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:52.609025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b9d9a-0004-486a-9704-24876cd67284 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:52.658284 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106a5e9-2cfc-4971-b2d5-fd73f6a400ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:52.659433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106a5e9-2cfc-4971-b2d5-fd73f6a400ed None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:52.659793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106a5e9-2cfc-4971-b2d5-fd73f6a400ed None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:52.660140 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106a5e9-2cfc-4971-b2d5-fd73f6a400ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:52.720493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e0159-5a92-4d28-8b74-98fcec172977 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:52.753316 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106a5e9-2cfc-4971-b2d5-fd73f6a400ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:52.755225 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106a5e9-2cfc-4971-b2d5-fd73f6a400ed None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:52.757696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:52.773900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e05221-19e7-460d-88ff-652e43e92460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:52.774914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e05221-19e7-460d-88ff-652e43e92460 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:52.775325 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e05221-19e7-460d-88ff-652e43e92460 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:52.775746 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e05221-19e7-460d-88ff-652e43e92460 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:52.776482 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e05221-19e7-460d-88ff-652e43e92460 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:52.776855 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e05221-19e7-460d-88ff-652e43e92460 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:52.779075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e05221-19e7-460d-88ff-652e43e92460 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:52.805006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 260/525] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:00:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:52.805688 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e05221-19e7-460d-88ff-652e43e92460 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:52.916988 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 266/526] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:00:52.918447 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106a5e9-2cfc-4971-b2d5-fd73f6a400ed None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:52.940737 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:52.958689 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e05221-19e7-460d-88ff-652e43e92460 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:53.004859 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88d044-e295-4aef-9961-f71b2e73da4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:53.006059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88d044-e295-4aef-9961-f71b2e73da4c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:53.006371 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88d044-e295-4aef-9961-f71b2e73da4c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:53.006861 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88d044-e295-4aef-9961-f71b2e73da4c None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:53.007799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88d044-e295-4aef-9961-f71b2e73da4c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:53.008195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88d044-e295-4aef-9961-f71b2e73da4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:53.009873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88d044-e295-4aef-9961-f71b2e73da4c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:53.038283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 261/527] 10.222.0.26 () {56 vars in 1139 bytes} [Tue Sep 6 16:00:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:53.038283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88d044-e295-4aef-9961-f71b2e73da4c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:53.073015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106a5e9-2cfc-4971-b2d5-fd73f6a400ed None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:53.075345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:53.135202 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882b0c6-ea5f-4828-9f25-193f92cf98d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:53.136112 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882b0c6-ea5f-4828-9f25-193f92cf98d6 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:53.136614 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882b0c6-ea5f-4828-9f25-193f92cf98d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:53.137085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882b0c6-ea5f-4828-9f25-193f92cf98d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:53.138377 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882b0c6-ea5f-4828-9f25-193f92cf98d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:53.138963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882b0c6-ea5f-4828-9f25-193f92cf98d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:53.141743 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882b0c6-ea5f-4828-9f25-193f92cf98d6 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:53.166499 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88d044-e295-4aef-9961-f71b2e73da4c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:53.174601 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 267/528] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:00:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:53.174601 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882b0c6-ea5f-4828-9f25-193f92cf98d6 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:53.235047 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba26d9e-7007-4f08-8b13-a9029815a05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:53.236666 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba26d9e-7007-4f08-8b13-a9029815a05f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:53.237340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba26d9e-7007-4f08-8b13-a9029815a05f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:53.237994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba26d9e-7007-4f08-8b13-a9029815a05f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:53.239404 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba26d9e-7007-4f08-8b13-a9029815a05f None admin] RBAC: Adding query filter params (role.id=56d621cf28924933b6356795a6f8ba23, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=e50ef7db27744c7ebd2ec95cecc9d1a8) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:53.240242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba26d9e-7007-4f08-8b13-a9029815a05f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:53.242963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba26d9e-7007-4f08-8b13-a9029815a05f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:53.252471 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 262/529] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:00:53 2022] GET /identity/v3/role_assignments?user.id=e50ef7db27744c7ebd2ec95cecc9d1a8&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=56d621cf28924933b6356795a6f8ba23 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:53.253708 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba26d9e-7007-4f08-8b13-a9029815a05f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:53.275283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:53.356430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882b0c6-ea5f-4828-9f25-193f92cf98d6 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:53.418692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1905d29-806b-47fe-baf4-28eee350c255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:53.420263 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1905d29-806b-47fe-baf4-28eee350c255 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:53.420796 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1905d29-806b-47fe-baf4-28eee350c255 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:53.421281 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1905d29-806b-47fe-baf4-28eee350c255 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:53.445413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1905d29-806b-47fe-baf4-28eee350c255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:53.448452 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1905d29-806b-47fe-baf4-28eee350c255 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:53.459197 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba26d9e-7007-4f08-8b13-a9029815a05f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:53.469795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-b1905d29-806b-47fe-baf4-28eee350c255 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:53.469795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:53.469795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:53.469795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:53.469795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:53.469795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:53.469795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:53.469795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:53.469795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:53.469795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:53.469795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:53.469795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:53.469795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:00:53.469795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:00:53.469795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:00:53.469795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:00:53.471688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:53.471688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:53.471688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:53.471688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:53.471688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:53.471688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:53.471688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:53.471688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:53.471688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:53.471688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:53.471688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:53.471688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:53.471688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:53.471688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:53.471688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:53.471688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:53.471688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:00:53.473383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:00:53.473383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:53.473383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:53.473383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 268/530] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:53.473383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1905d29-806b-47fe-baf4-28eee350c255 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:53.492917 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:53.541354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715e4e0-e960-41e3-9f23-c0836f100f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:53.542608 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715e4e0-e960-41e3-9f23-c0836f100f3f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:53.543364 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715e4e0-e960-41e3-9f23-c0836f100f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:53.543773 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715e4e0-e960-41e3-9f23-c0836f100f3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:53.627232 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1905d29-806b-47fe-baf4-28eee350c255 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:53.643639 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715e4e0-e960-41e3-9f23-c0836f100f3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:53.646650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715e4e0-e960-41e3-9f23-c0836f100f3f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:53.667323 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:53.669153 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729903d-b8e7-4286-b7b6-392ff7ac5276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:53.670066 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729903d-b8e7-4286-b7b6-392ff7ac5276 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:53.670499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729903d-b8e7-4286-b7b6-392ff7ac5276 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:53.671007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729903d-b8e7-4286-b7b6-392ff7ac5276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:53.671988 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729903d-b8e7-4286-b7b6-392ff7ac5276 None admin] RBAC: Adding query filter params (role.id=56d621cf28924933b6356795a6f8ba23, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=6afd4f6ffdae4b16aa5ba757e68c18c4) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:53.672471 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729903d-b8e7-4286-b7b6-392ff7ac5276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:53.674130 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729903d-b8e7-4286-b7b6-392ff7ac5276 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:53.691205 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 269/531] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:00:53 2022] GET /identity/v3/role_assignments?user.id=6afd4f6ffdae4b16aa5ba757e68c18c4&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=56d621cf28924933b6356795a6f8ba23 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:53.692515 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729903d-b8e7-4286-b7b6-392ff7ac5276 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:53.873428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 263/532] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:00:53.887728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:53.889755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715e4e0-e960-41e3-9f23-c0836f100f3f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:53.924030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729903d-b8e7-4286-b7b6-392ff7ac5276 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:53.993907 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91479f-51cb-439a-a00a-b8635d505cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:53.994964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91479f-51cb-439a-a00a-b8635d505cfc None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:53.995467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91479f-51cb-439a-a00a-b8635d505cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:53.996004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91479f-51cb-439a-a00a-b8635d505cfc None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:53.997148 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91479f-51cb-439a-a00a-b8635d505cfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:53.997687 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91479f-51cb-439a-a00a-b8635d505cfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:53.999386 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91479f-51cb-439a-a00a-b8635d505cfc None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:54.035181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 270/533] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:54.039267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91479f-51cb-439a-a00a-b8635d505cfc None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:54.055402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715e4e0-e960-41e3-9f23-c0836f100f3f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:54.067463 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:54.096614 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8353f79-d5f1-48ab-a65c-a25f499f49a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:54.097068 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8353f79-d5f1-48ab-a65c-a25f499f49a7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:54.097174 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8353f79-d5f1-48ab-a65c-a25f499f49a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:54.097458 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8353f79-d5f1-48ab-a65c-a25f499f49a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:54.113143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8353f79-d5f1-48ab-a65c-a25f499f49a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:54.114641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8353f79-d5f1-48ab-a65c-a25f499f49a7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:54.156544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-a8353f79-d5f1-48ab-a65c-a25f499f49a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:54.156544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:54.156544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:54.156544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:54.156544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:54.156544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:54.156544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:54.156544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:54.156544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:54.156544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:54.156544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:54.156544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:54.156544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:00:54.156544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:00:54.156544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:00:54.156544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:00:54.157500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:54.157500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:54.157500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:54.157500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:54.157500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:54.157500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:54.157500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:54.157500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:54.157500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:54.157500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:54.157500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:54.157500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:54.157500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:54.157500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:54.157500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:54.157500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:54.157500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:00:54.158320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:00:54.158320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:54.158320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:54.158320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 264/534] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:54.174068 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8353f79-d5f1-48ab-a65c-a25f499f49a7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:54.282439 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91479f-51cb-439a-a00a-b8635d505cfc None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:54.387421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1102dc-30db-4e8e-aab2-170e1c682af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:54.388087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1102dc-30db-4e8e-aab2-170e1c682af3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:54.388371 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1102dc-30db-4e8e-aab2-170e1c682af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:54.388632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1102dc-30db-4e8e-aab2-170e1c682af3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:54.408042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1102dc-30db-4e8e-aab2-170e1c682af3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:54.409461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1102dc-30db-4e8e-aab2-170e1c682af3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:54.411855 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:54.428424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-eb1102dc-30db-4e8e-aab2-170e1c682af3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:00:54.428424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:54.428424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:54.428424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:54.428424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:54.428424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:54.428424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:54.428424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:54.428424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:54.428424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:54.428424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:54.428424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:54.428424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:00:54.428424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:00:54.428424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:00:54.429740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:00:54.429740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:54.429740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:54.429740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:54.429740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:54.429740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:54.429740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:54.429740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:54.429740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:54.429740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:54.429740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:54.429740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:54.429740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:54.429740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:54.429740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:54.429740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:54.429740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:54.430584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:00:54.430584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:00:54.430584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:00:54.430584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:00:54.430584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:00:54.430584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:00:54.430584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:00:54.430584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:54.439122 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 271/535] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:00:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:00:54.442650 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1102dc-30db-4e8e-aab2-170e1c682af3 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:54.451101 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8353f79-d5f1-48ab-a65c-a25f499f49a7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:54.532441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cc780-249e-4833-a658-3921836cbc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:54.534145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cc780-249e-4833-a658-3921836cbc25 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:54.535132 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cc780-249e-4833-a658-3921836cbc25 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:54.535600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cc780-249e-4833-a658-3921836cbc25 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:54.537293 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cc780-249e-4833-a658-3921836cbc25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:54.537827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cc780-249e-4833-a658-3921836cbc25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:54.539556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cc780-249e-4833-a658-3921836cbc25 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:54.567534 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 265/536] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:54.570305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c93ff0-e01b-40aa-9b92-fcc6b0d33c60 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:54.573420 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c93ff0-e01b-40aa-9b92-fcc6b0d33c60 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:54.576892 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c93ff0-e01b-40aa-9b92-fcc6b0d33c60 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:54.579861 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 266/537] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:54 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:54.585019 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c93ff0-e01b-40aa-9b92-fcc6b0d33c60 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:54.627572 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:54.717725 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1102dc-30db-4e8e-aab2-170e1c682af3 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:54.769897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c93ff0-e01b-40aa-9b92-fcc6b0d33c60 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:54.788262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd9490-e651-4179-8005-b15f190186cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:54.790249 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd9490-e651-4179-8005-b15f190186cc None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:54.790775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd9490-e651-4179-8005-b15f190186cc None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:54.791641 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd9490-e651-4179-8005-b15f190186cc None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:54.793842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd9490-e651-4179-8005-b15f190186cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:54.794427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd9490-e651-4179-8005-b15f190186cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:54.797582 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd9490-e651-4179-8005-b15f190186cc None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:54.814769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5370b-1899-4091-b081-aaed63945ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:54.815409 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5370b-1899-4091-b081-aaed63945ecf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:54.815680 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5370b-1899-4091-b081-aaed63945ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:54.815971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5370b-1899-4091-b081-aaed63945ecf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:54.825806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5370b-1899-4091-b081-aaed63945ecf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:54.827242 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 272/538] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:00:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:54.829483 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e177918-afc1-414f-b675-423a049b0af7 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:54.829984 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e177918-afc1-414f-b675-423a049b0af7 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:54.830477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e177918-afc1-414f-b675-423a049b0af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:54.830979 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5370b-1899-4091-b081-aaed63945ecf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:54.847573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-75d5370b-1899-4091-b081-aaed63945ecf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:00:54.847573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:54.847573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:54.847573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:54.847573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:54.847573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:54.847573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:54.847573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:54.847573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:54.847573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:54.847573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:54.847573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:54.847573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:00:54.847573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:00:54.847573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:00:54.849801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:00:54.849801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:54.849801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:54.849801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:54.849801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:54.849801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:54.849801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:54.849801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:54.849801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:54.849801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:54.849801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:54.849801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:54.849801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:54.849801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:54.849801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:54.849801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:54.849801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:54.853077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:00:54.853077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:00:54.853077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:00:54.853077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:00:54.853077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:00:54.853077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:00:54.853077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:00:54.853077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:54.854348 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 267/539] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:00:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:54.862694 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5370b-1899-4091-b081-aaed63945ecf None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:54.864581 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8e177918-afc1-414f-b675-423a049b0af7 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:54.868333 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8e177918-afc1-414f-b675-423a049b0af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:54.970452 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:54.992502 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 273/540] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:54.994038 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:54.994369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e177918-afc1-414f-b675-423a049b0af7 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:55.064425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5370b-1899-4091-b081-aaed63945ecf None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:55.154078 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7efc7-1938-40be-8824-efbaaa8ed08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:55.155165 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7efc7-1938-40be-8824-efbaaa8ed08a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:55.155840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7efc7-1938-40be-8824-efbaaa8ed08a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:55.157150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7efc7-1938-40be-8824-efbaaa8ed08a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:55.158315 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7efc7-1938-40be-8824-efbaaa8ed08a None admin] RBAC: Adding query filter params (role.id=56d621cf28924933b6356795a6f8ba23, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=7849ef049ffd4495909f997735af8782) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:55.160515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7efc7-1938-40be-8824-efbaaa8ed08a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:55.162969 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7efc7-1938-40be-8824-efbaaa8ed08a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:55.179201 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 268/541] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:00:54 2022] GET /identity/v3/role_assignments?user.id=7849ef049ffd4495909f997735af8782&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=56d621cf28924933b6356795a6f8ba23 => generated 259 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:55.248340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e177918-afc1-414f-b675-423a049b0af7 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:55.295272 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069df5f3-cef5-431b-8f32-18cb33549426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:55.296381 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069df5f3-cef5-431b-8f32-18cb33549426 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:55.296863 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069df5f3-cef5-431b-8f32-18cb33549426 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:55.297337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069df5f3-cef5-431b-8f32-18cb33549426 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:55.298273 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069df5f3-cef5-431b-8f32-18cb33549426 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:55.298823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069df5f3-cef5-431b-8f32-18cb33549426 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:55.300853 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069df5f3-cef5-431b-8f32-18cb33549426 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:55.320921 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 274/542] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:00:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:55.330851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7efc7-1938-40be-8824-efbaaa8ed08a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:55.472157 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7efc7-1938-40be-8824-efbaaa8ed08a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:55.507577 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069df5f3-cef5-431b-8f32-18cb33549426 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:55.521937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54332f2f-3bb0-4fc6-9388-b2353e959bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:55.527647 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54332f2f-3bb0-4fc6-9388-b2353e959bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:55.528285 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54332f2f-3bb0-4fc6-9388-b2353e959bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:55.528778 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54332f2f-3bb0-4fc6-9388-b2353e959bc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:55.532222 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54332f2f-3bb0-4fc6-9388-b2353e959bc2 None admin] RBAC: Adding query filter params (role.id=56d621cf28924933b6356795a6f8ba23, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=989beaca146d4f9b950f333bd3567e6a) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:55.532934 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54332f2f-3bb0-4fc6-9388-b2353e959bc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:55.535174 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54332f2f-3bb0-4fc6-9388-b2353e959bc2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:55.549392 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 269/543] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:00:55 2022] GET /identity/v3/role_assignments?user.id=989beaca146d4f9b950f333bd3567e6a&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=56d621cf28924933b6356795a6f8ba23 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:55.597935 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:55.662115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069df5f3-cef5-431b-8f32-18cb33549426 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:55.701965 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3ace0-f26b-4694-a159-7d3e24b48505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:55.702623 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3ace0-f26b-4694-a159-7d3e24b48505 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:55.702937 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3ace0-f26b-4694-a159-7d3e24b48505 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:55.703209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3ace0-f26b-4694-a159-7d3e24b48505 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:55.716421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3ace0-f26b-4694-a159-7d3e24b48505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:55.722064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-d0a3ace0-f26b-4694-a159-7d3e24b48505 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:00:55.722064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:55.722064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:55.722064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:55.722064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:55.722064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:55.722064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:55.722064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:55.722064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:55.722064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:55.722064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:55.722064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:55.722064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:55.722064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:55.722064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:55.722064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:55.723051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:55.723051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:55.723051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:55.723051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:55.723051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:55.723051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:55.723051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:55.723051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:55.723051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:55.723051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:55.723051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:55.723051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:55.723051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:55.723051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:55.723051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:55.723051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:55.723867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:55.723867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:55.723867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:55.723867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:55.723867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:55.723867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:55.723867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:55.723867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:55.723867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:55.723867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:55.723867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:00:55.723867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:55.723867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 275/544] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:00:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:00:55.732701 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54332f2f-3bb0-4fc6-9388-b2353e959bc2 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:55.763347 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:55.884330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54332f2f-3bb0-4fc6-9388-b2353e959bc2 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:55.928885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89718b08-fc62-42e2-93f4-14605af9dc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:55.930350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89718b08-fc62-42e2-93f4-14605af9dc54 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:55.931168 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89718b08-fc62-42e2-93f4-14605af9dc54 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:55.931904 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89718b08-fc62-42e2-93f4-14605af9dc54 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:55.933462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89718b08-fc62-42e2-93f4-14605af9dc54 None admin] RBAC: Adding query filter params (name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:55.934291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89718b08-fc62-42e2-93f4-14605af9dc54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:55.937024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89718b08-fc62-42e2-93f4-14605af9dc54 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:55.960442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 270/545] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:00:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:00:55.968954 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3ace0-f26b-4694-a159-7d3e24b48505 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:56.107815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3ace0-f26b-4694-a159-7d3e24b48505 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:56.118614 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:56.157318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57606d5a-cf72-4f52-af25-515fff87e453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:56.157976 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57606d5a-cf72-4f52-af25-515fff87e453 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:56.158243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57606d5a-cf72-4f52-af25-515fff87e453 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:56.158508 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57606d5a-cf72-4f52-af25-515fff87e453 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:56.177197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57606d5a-cf72-4f52-af25-515fff87e453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:56.178837 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57606d5a-cf72-4f52-af25-515fff87e453 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:56.195987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-57606d5a-cf72-4f52-af25-515fff87e453 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:56.195987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:56.195987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:56.195987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:56.195987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:56.195987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:56.195987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:56.195987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:56.195987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:56.195987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:56.195987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:56.195987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:56.195987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:00:56.195987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:00:56.195987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:00:56.195987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:00:56.197786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:56.197786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:56.197786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:56.197786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:56.197786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:56.197786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:56.197786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:56.197786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:56.197786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:56.197786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:56.197786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:56.197786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:56.197786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:56.197786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:56.197786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:56.197786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:56.197786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:00:56.199519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:00:56.199519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:56.199519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:56.199519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 276/546] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:56.205195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89718b08-fc62-42e2-93f4-14605af9dc54 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:56.353780 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:56.429746 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89718b08-fc62-42e2-93f4-14605af9dc54 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:56.503931 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7bcf2-4f93-435f-b4ce-54c886cf82dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:56.505544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7bcf2-4f93-435f-b4ce-54c886cf82dc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:56.506320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7bcf2-4f93-435f-b4ce-54c886cf82dc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:56.507060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7bcf2-4f93-435f-b4ce-54c886cf82dc None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:56.508374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7bcf2-4f93-435f-b4ce-54c886cf82dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:56.509119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7bcf2-4f93-435f-b4ce-54c886cf82dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:56.511710 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7bcf2-4f93-435f-b4ce-54c886cf82dc None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:56.536314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 271/547] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:56.547150 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57606d5a-cf72-4f52-af25-515fff87e453 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:56.593546 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480e65a-825d-430f-bb53-825c467a6154 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:56.594046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480e65a-825d-430f-bb53-825c467a6154 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:56.594540 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480e65a-825d-430f-bb53-825c467a6154 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:56.597759 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 272/548] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:00:56.607846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21660378-3801-4563-bbfc-00ddd286f626 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:56.608345 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21660378-3801-4563-bbfc-00ddd286f626 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:56.608843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21660378-3801-4563-bbfc-00ddd286f626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:56.642188 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-21660378-3801-4563-bbfc-00ddd286f626 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:56.644613 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-21660378-3801-4563-bbfc-00ddd286f626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:56.743139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57606d5a-cf72-4f52-af25-515fff87e453 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:56.795830 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 273/549] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:56.798604 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e48156-5ba8-415f-8c83-fb844871594f None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:56.799408 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e48156-5ba8-415f-8c83-fb844871594f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:56.800354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e48156-5ba8-415f-8c83-fb844871594f None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:56.809784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 274/550] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:56.812542 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6009f-d10f-4c30-a36b-11b860d767d2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:56.813399 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6009f-d10f-4c30-a36b-11b860d767d2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:56.814318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6009f-d10f-4c30-a36b-11b860d767d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:56.856025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2b01e-2e69-4d63-b1b5-ce692560f68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:56.857118 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2b01e-2e69-4d63-b1b5-ce692560f68d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:56.857609 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2b01e-2e69-4d63-b1b5-ce692560f68d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:56.858178 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2b01e-2e69-4d63-b1b5-ce692560f68d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:56.863568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-70c6009f-d10f-4c30-a36b-11b860d767d2 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:56.871304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-70c6009f-d10f-4c30-a36b-11b860d767d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:56.885106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2b01e-2e69-4d63-b1b5-ce692560f68d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:56.887826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2b01e-2e69-4d63-b1b5-ce692560f68d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:56.931684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-5aa2b01e-2e69-4d63-b1b5-ce692560f68d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:00:56.931684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:56.931684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:56.931684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:56.931684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:56.931684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:56.931684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:56.931684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:56.931684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:56.931684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:56.931684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:56.931684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:56.931684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:00:56.931684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:00:56.931684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:00:56.931684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:00:56.933474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:56.933474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:56.933474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:00:56.933474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:00:56.933474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:00:56.933474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:00:56.933474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:56.933474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:56.933474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:56.933474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:56.933474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:56.933474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:56.933474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:56.933474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:56.933474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:56.933474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:56.933474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:56.933474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:56.935336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:56.935336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:56.935336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:00:56.935336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:00:56.935336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:00:56.935336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:00:56.935336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:00:56.935336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:00:56.935336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:00:56.935336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:56.945868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 277/551] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:00:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:56.966726 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2b01e-2e69-4d63-b1b5-ce692560f68d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:57.036972 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:57.069669 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:57.107182 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 275/552] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:57.150584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2b01e-2e69-4d63-b1b5-ce692560f68d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:57.209701 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cd073-f8a4-4df2-9812-477cdb5ba419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:57.210325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cd073-f8a4-4df2-9812-477cdb5ba419 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:57.210602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cd073-f8a4-4df2-9812-477cdb5ba419 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:57.210893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cd073-f8a4-4df2-9812-477cdb5ba419 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:57.211653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990cd073-f8a4-4df2-9812-477cdb5ba419 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:57.211958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990cd073-f8a4-4df2-9812-477cdb5ba419 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:57.213453 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990cd073-f8a4-4df2-9812-477cdb5ba419 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:57.257247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 278/553] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Sep 6 16:00:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:57.265615 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6009f-d10f-4c30-a36b-11b860d767d2 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:57.386787 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cd073-f8a4-4df2-9812-477cdb5ba419 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:57.569779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6009f-d10f-4c30-a36b-11b860d767d2 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:57.576599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:57.625898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cd073-f8a4-4df2-9812-477cdb5ba419 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:57.681501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3dc183-1bae-4e57-86a8-70504d6fe7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:57.682423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dc183-1bae-4e57-86a8-70504d6fe7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:57.682938 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dc183-1bae-4e57-86a8-70504d6fe7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:57.683410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dc183-1bae-4e57-86a8-70504d6fe7b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:57.700547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f1702-326f-4a78-b3fb-19c8a832d8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:57.702179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f1702-326f-4a78-b3fb-19c8a832d8eb None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:57.703043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f1702-326f-4a78-b3fb-19c8a832d8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:57.703824 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f1702-326f-4a78-b3fb-19c8a832d8eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:57.725163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f1702-326f-4a78-b3fb-19c8a832d8eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:57.735208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-a96f1702-326f-4a78-b3fb-19c8a832d8eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:00:57.735208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:57.735208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:57.735208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:57.735208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:57.735208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:57.735208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:57.735208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:57.735208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:57.735208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:57.735208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:57.735208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:57.735208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:57.735208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:57.735208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:57.735208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:57.736946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:57.736946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:57.736946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:57.736946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:57.736946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:57.736946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:57.736946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:57.736946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:57.736946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:57.736946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:57.736946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:57.736946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:57.736946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:57.736946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:57.736946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:57.736946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:57.738427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:57.738427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:57.738427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:57.738427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:57.738427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:57.738427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:57.738427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:57.738427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:57.738427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:57.738427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:57.738427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:00:57.738427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:57.743171 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 279/554] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:00:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:00:57.751734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f1702-326f-4a78-b3fb-19c8a832d8eb None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:57.820928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:57.841192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3dc183-1bae-4e57-86a8-70504d6fe7b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:57.844465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3dc183-1bae-4e57-86a8-70504d6fe7b9 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:57.967731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f1702-326f-4a78-b3fb-19c8a832d8eb None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:58.037766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e318c9-12a7-4fd3-a03d-c5177cdbdf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:58.039434 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e318c9-12a7-4fd3-a03d-c5177cdbdf9f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:58.040206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e318c9-12a7-4fd3-a03d-c5177cdbdf9f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:58.041069 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e318c9-12a7-4fd3-a03d-c5177cdbdf9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:58.058482 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:58.070466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e318c9-12a7-4fd3-a03d-c5177cdbdf9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:58.083025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-42e318c9-12a7-4fd3-a03d-c5177cdbdf9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:00:58.083025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:58.083025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:58.083025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:58.083025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:58.083025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:58.083025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:58.083025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:58.083025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:58.083025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:58.083025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:58.083025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:58.083025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:58.083025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:58.083025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:58.083025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:58.085376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:58.085376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:58.085376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:58.085376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:58.085376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:58.085376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:58.085376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:58.085376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:58.085376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:58.085376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:58.085376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:58.085376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:58.085376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:58.085376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:58.085376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:58.085376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:58.087081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:58.087081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:58.087081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:58.087081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:58.087081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:58.087081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:58.087081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:58.087081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:58.087081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:58.087081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:58.087081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:00:58.087081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:58.098703 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 280/556] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:00:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:58.098703 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e318c9-12a7-4fd3-a03d-c5177cdbdf9f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:58.099419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 276/556] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:00:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 828 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:00:58.101379 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3dc183-1bae-4e57-86a8-70504d6fe7b9 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:58.299656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:58.310215 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e318c9-12a7-4fd3-a03d-c5177cdbdf9f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:58.315776 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3dc183-1bae-4e57-86a8-70504d6fe7b9 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:58.363200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21762827-d948-4f1c-89a2-514d55daf959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:58.363947 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21762827-d948-4f1c-89a2-514d55daf959 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:58.364305 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21762827-d948-4f1c-89a2-514d55daf959 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:58.364713 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21762827-d948-4f1c-89a2-514d55daf959 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:58.365628 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21762827-d948-4f1c-89a2-514d55daf959 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:58.366048 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21762827-d948-4f1c-89a2-514d55daf959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:58.367642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21762827-d948-4f1c-89a2-514d55daf959 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:58.368994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ce280-24cd-4a8f-831d-77100c5cdc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:58.369805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ce280-24cd-4a8f-831d-77100c5cdc00 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:58.370141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ce280-24cd-4a8f-831d-77100c5cdc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:58.370585 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ce280-24cd-4a8f-831d-77100c5cdc00 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:58.371390 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ce280-24cd-4a8f-831d-77100c5cdc00 None admin] RBAC: Adding query filter params (name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:58.371969 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ce280-24cd-4a8f-831d-77100c5cdc00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:58.373222 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ce280-24cd-4a8f-831d-77100c5cdc00 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:58.377524 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 281/557] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:00:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:58.378325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21762827-d948-4f1c-89a2-514d55daf959 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:58.390883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 277/558] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:00:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:58.392943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3720e6-243f-4fbc-af48-0679b954232e None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:58.393420 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3720e6-243f-4fbc-af48-0679b954232e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:58.393888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3720e6-243f-4fbc-af48-0679b954232e None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:58.396484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 278/559] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:58.398709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcd08b-a0a4-4866-b013-0f2fc400fe74 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:58.399216 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcd08b-a0a4-4866-b013-0f2fc400fe74 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:58.399694 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcd08b-a0a4-4866-b013-0f2fc400fe74 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:58.402152 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 279/560] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:00:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:00:58.403815 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcd08b-a0a4-4866-b013-0f2fc400fe74 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:58.457901 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:58.518433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21762827-d948-4f1c-89a2-514d55daf959 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:58.540244 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcd08b-a0a4-4866-b013-0f2fc400fe74 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:58.562350 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b50de-987d-4343-a12b-17cc57c42310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:58.563048 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b50de-987d-4343-a12b-17cc57c42310 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:58.563326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b50de-987d-4343-a12b-17cc57c42310 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:58.563595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b50de-987d-4343-a12b-17cc57c42310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:58.575855 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b50de-987d-4343-a12b-17cc57c42310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:58.577356 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b50de-987d-4343-a12b-17cc57c42310 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:58.581900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a2100-158c-4d80-8871-67605e5c9347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:58.582555 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a2100-158c-4d80-8871-67605e5c9347 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:58.582848 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a2100-158c-4d80-8871-67605e5c9347 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:58.583120 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a2100-158c-4d80-8871-67605e5c9347 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:58.593842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-c50b50de-987d-4343-a12b-17cc57c42310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:58.593842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:58.593842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:58.593842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:58.593842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:58.593842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:58.593842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:58.593842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:58.593842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:58.593842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:58.593842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:58.593842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:58.593842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:00:58.593842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:00:58.593842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:00:58.593842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:00:58.595307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:58.595307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:58.595307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:58.595307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:58.595307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:58.595307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:58.595307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:58.595307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:58.595307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:58.595307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:58.595307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:58.595307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:58.595307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:58.595307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:58.595307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:58.595307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:58.595307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:00:58.597114 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:00:58.597114 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:58.597114 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:58.597114 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 282/561] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:58.597561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a2100-158c-4d80-8871-67605e5c9347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:58.597692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b50de-987d-4343-a12b-17cc57c42310 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:58.597821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a2100-158c-4d80-8871-67605e5c9347 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:58.623017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-8d3a2100-158c-4d80-8871-67605e5c9347 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:58.623017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:58.623017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:58.623017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:58.623017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:58.623017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:58.623017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:58.623017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:58.623017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:58.623017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:58.623017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:58.623017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:58.623017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:00:58.623017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:00:58.623017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:00:58.623017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:00:58.624561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:58.624561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:58.624561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:58.624561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:58.624561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:58.624561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:58.624561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:58.624561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:58.624561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:58.624561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:58.624561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:58.624561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:58.624561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:58.624561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:58.624561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:58.624561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:58.624561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:00:58.625455 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:00:58.625455 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:58.625455 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:58.625455 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 280/562] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:58.625455 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea13ba-235a-4797-965f-3ea8a42c9f36 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:58.625740 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea13ba-235a-4797-965f-3ea8a42c9f36 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:58.625860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea13ba-235a-4797-965f-3ea8a42c9f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:58.639295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:58.645310 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-67ea13ba-235a-4797-965f-3ea8a42c9f36 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:58.647129 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-67ea13ba-235a-4797-965f-3ea8a42c9f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:58.716112 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b50de-987d-4343-a12b-17cc57c42310 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:58.724286 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 281/563] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:58.725580 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb698f0a-b8b2-4792-b25b-23a79a19d893 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:58.725950 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb698f0a-b8b2-4792-b25b-23a79a19d893 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:58.726227 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb698f0a-b8b2-4792-b25b-23a79a19d893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:58.739637 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:58.752099 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-fb698f0a-b8b2-4792-b25b-23a79a19d893 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:00:58.753316 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fb698f0a-b8b2-4792-b25b-23a79a19d893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:00:58.756497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34fe2c-fc5b-414d-97fb-16fbb9c214f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:58.757914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34fe2c-fc5b-414d-97fb-16fbb9c214f1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:58.758190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34fe2c-fc5b-414d-97fb-16fbb9c214f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:58.758468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34fe2c-fc5b-414d-97fb-16fbb9c214f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:58.770653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34fe2c-fc5b-414d-97fb-16fbb9c214f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:58.773162 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34fe2c-fc5b-414d-97fb-16fbb9c214f1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:58.790318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-eb34fe2c-fc5b-414d-97fb-16fbb9c214f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:58.790318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:58.790318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:58.790318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:58.790318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:58.790318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:58.790318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:58.790318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:58.790318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:58.790318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:58.790318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:58.790318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:58.790318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:00:58.790318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:00:58.790318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:00:58.790318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:00:58.791343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:58.791343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:58.791343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:58.791343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:58.791343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:58.791343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:58.791343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:58.791343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:58.791343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:58.791343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:58.791343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:58.791343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:58.791343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:58.791343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:58.791343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:58.791343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:58.791343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:00:58.792191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:00:58.792191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:00:58.792191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:58.792191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 283/564] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:58.792852 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34fe2c-fc5b-414d-97fb-16fbb9c214f1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:58.861494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 282/565] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:00:58.862706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb698f0a-b8b2-4792-b25b-23a79a19d893 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:58.973284 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34fe2c-fc5b-414d-97fb-16fbb9c214f1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:59.017308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:59.046203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25d5c3-e28c-4802-af9d-2b3382a98732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:59.047280 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25d5c3-e28c-4802-af9d-2b3382a98732 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:59.047778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25d5c3-e28c-4802-af9d-2b3382a98732 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:59.048314 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25d5c3-e28c-4802-af9d-2b3382a98732 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:59.049471 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25d5c3-e28c-4802-af9d-2b3382a98732 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:59.049974 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25d5c3-e28c-4802-af9d-2b3382a98732 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:59.052963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25d5c3-e28c-4802-af9d-2b3382a98732 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:59.079670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 284/566] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:59.079670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25d5c3-e28c-4802-af9d-2b3382a98732 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:59.092362 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb698f0a-b8b2-4792-b25b-23a79a19d893 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:59.123190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:59.153092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facffdf6-2965-486f-bc74-fd172bcd3b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:59.154072 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facffdf6-2965-486f-bc74-fd172bcd3b95 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:59.154479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facffdf6-2965-486f-bc74-fd172bcd3b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:59.155000 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facffdf6-2965-486f-bc74-fd172bcd3b95 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:59.156072 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facffdf6-2965-486f-bc74-fd172bcd3b95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:59.156619 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facffdf6-2965-486f-bc74-fd172bcd3b95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:59.159250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facffdf6-2965-486f-bc74-fd172bcd3b95 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:59.182716 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 283/567] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:59.183574 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facffdf6-2965-486f-bc74-fd172bcd3b95 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:59.241519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25d5c3-e28c-4802-af9d-2b3382a98732 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:59.284743 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15220a23-5dfa-4f4f-bc42-236015735b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:59.285463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15220a23-5dfa-4f4f-bc42-236015735b8a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:59.285737 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15220a23-5dfa-4f4f-bc42-236015735b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:59.286002 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15220a23-5dfa-4f4f-bc42-236015735b8a None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:59.286744 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15220a23-5dfa-4f4f-bc42-236015735b8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:59.287044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15220a23-5dfa-4f4f-bc42-236015735b8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:59.288431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15220a23-5dfa-4f4f-bc42-236015735b8a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:59.303685 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 285/568] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:59.305105 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15220a23-5dfa-4f4f-bc42-236015735b8a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:59.335287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facffdf6-2965-486f-bc74-fd172bcd3b95 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:59.367233 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:59.380444 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b3363-2a99-402b-a9a1-551349b934e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:59.381476 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b3363-2a99-402b-a9a1-551349b934e3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:59.381760 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b3363-2a99-402b-a9a1-551349b934e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:59.382053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b3363-2a99-402b-a9a1-551349b934e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:59.392836 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b3363-2a99-402b-a9a1-551349b934e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:59.399322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-386b3363-2a99-402b-a9a1-551349b934e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:00:59.399322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:59.399322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:59.399322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:59.399322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:59.399322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:59.399322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:59.399322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:59.399322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:59.399322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:59.399322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:59.399322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:59.399322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:59.399322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:59.399322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:59.399322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:59.401728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:59.401728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:59.401728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:59.401728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:59.401728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:59.401728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:59.401728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:59.401728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:59.401728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:59.401728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:59.401728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:59.401728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:59.401728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:59.401728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:59.401728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:59.401728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:59.402548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:59.402548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:59.402548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:59.402548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:59.402548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:59.402548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:59.402548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:59.402548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:59.402548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:59.402548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:59.402548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:00:59.402548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:59.402548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 284/569] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:00:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:59.402548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b3363-2a99-402b-a9a1-551349b934e3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:59.413171 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:59.431901 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15220a23-5dfa-4f4f-bc42-236015735b8a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:59.467794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b3c09-8a9c-4a15-941a-76ce6e357e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:59.468429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b3c09-8a9c-4a15-941a-76ce6e357e41 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:59.468709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b3c09-8a9c-4a15-941a-76ce6e357e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:59.468981 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b3c09-8a9c-4a15-941a-76ce6e357e41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:59.476916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b3c09-8a9c-4a15-941a-76ce6e357e41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:59.478397 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b3c09-8a9c-4a15-941a-76ce6e357e41 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:59.490302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-174b3c09-8a9c-4a15-941a-76ce6e357e41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:00:59.490302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:59.490302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:59.490302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:59.490302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:59.490302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:59.490302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:59.490302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:59.490302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:59.490302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:59.490302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:59.490302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:59.490302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:00:59.490302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:00:59.490302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:00:59.492063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:00:59.492063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:59.492063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:59.492063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:59.492063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:59.492063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:59.492063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:59.492063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:59.492063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:59.492063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:59.492063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:59.492063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:59.492063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:59.492063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:59.492063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:59.492063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:59.492063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:59.493706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:00:59.493706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:00:59.493706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:00:59.493706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:00:59.493706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:00:59.493706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:00:59.493706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:00:59.493706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:59.493706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 286/570] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:00:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:59.493706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b3c09-8a9c-4a15-941a-76ce6e357e41 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:59.531273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b3363-2a99-402b-a9a1-551349b934e3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:59.570217 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e5242-5ebd-40b7-9bf7-2ed733a6c327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:59.570889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e5242-5ebd-40b7-9bf7-2ed733a6c327 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:59.571169 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e5242-5ebd-40b7-9bf7-2ed733a6c327 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:59.571442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e5242-5ebd-40b7-9bf7-2ed733a6c327 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:59.579556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e5242-5ebd-40b7-9bf7-2ed733a6c327 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:59.581013 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e5242-5ebd-40b7-9bf7-2ed733a6c327 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:59.602885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-1e4e5242-5ebd-40b7-9bf7-2ed733a6c327 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 16:00:59.602885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:59.602885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:59.602885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:59.602885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:59.602885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:59.602885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:59.602885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:59.602885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:59.602885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:59.602885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:59.602885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:59.602885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:00:59.602885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:00:59.602885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:00:59.602885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:00:59.604192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:59.604192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:59.604192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:00:59.604192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:00:59.604192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:00:59.604192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:00:59.604192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:59.604192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:59.604192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:59.604192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:59.604192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:59.604192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:59.604192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:59.604192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:59.604192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:59.604192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:59.604192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:59.604192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:59.605106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:59.605106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:59.605106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:00:59.605106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:00:59.605106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:00:59.605106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:00:59.605106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:00:59.605106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:00:59.605106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 16:00:59.605106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:59.605106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 285/571] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:00:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:59.605685 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e5242-5ebd-40b7-9bf7-2ed733a6c327 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:59.624460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b3c09-8a9c-4a15-941a-76ce6e357e41 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:59.661860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04928d-d7d7-4237-a534-66551d3c89f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:59.662411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04928d-d7d7-4237-a534-66551d3c89f7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:59.662741 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04928d-d7d7-4237-a534-66551d3c89f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:59.663014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04928d-d7d7-4237-a534-66551d3c89f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:59.674381 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04928d-d7d7-4237-a534-66551d3c89f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:00:59.680176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-fb04928d-d7d7-4237-a534-66551d3c89f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:00:59.680176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:59.680176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:59.680176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:59.680176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:59.680176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:59.680176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:59.680176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:59.680176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:59.680176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:59.680176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:59.680176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:59.680176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:00:59.680176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:00:59.680176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:00:59.680176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:00:59.681159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:00:59.681159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:00:59.681159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:00:59.681159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:00:59.681159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:59.681159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:59.681159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:59.681159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:59.681159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:59.681159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:59.681159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:59.681159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:59.681159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:59.681159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:59.681159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:59.681159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:59.682005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:59.682005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:59.682005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:59.682005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:59.682005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:00:59.682005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:00:59.682005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:00:59.682005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:00:59.682005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:00:59.682005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:00:59.682005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:00:59.682005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:00:59.682005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 287/572] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:00:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:59.682713 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04928d-d7d7-4237-a534-66551d3c89f7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:59.683959 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:59.687569 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:59.744804 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e5242-5ebd-40b7-9bf7-2ed733a6c327 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:59.786563 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bef2b-3076-44bf-aecb-cdb72e39b674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:59.787670 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bef2b-3076-44bf-aecb-cdb72e39b674 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:59.788186 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bef2b-3076-44bf-aecb-cdb72e39b674 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:59.788584 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bef2b-3076-44bf-aecb-cdb72e39b674 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:59.798261 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707bef2b-3076-44bf-aecb-cdb72e39b674 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:59.799791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707bef2b-3076-44bf-aecb-cdb72e39b674 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:59.807402 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04928d-d7d7-4237-a534-66551d3c89f7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:59.812660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-707bef2b-3076-44bf-aecb-cdb72e39b674 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:00:59.812660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:00:59.812660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:00:59.812660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:00:59.812660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:00:59.812660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:00:59.812660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:00:59.812660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:00:59.812660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:00:59.812660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:00:59.812660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:00:59.812660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:00:59.812660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:00:59.812660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:00:59.812660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:00:59.812660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:00:59.813638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:00:59.813638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:00:59.813638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:00:59.813638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:00:59.813638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:00:59.813638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:00:59.813638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:00:59.813638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:00:59.813638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:00:59.813638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:00:59.813638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:00:59.813638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:00:59.813638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:00:59.813638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:00:59.813638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:00:59.813638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:00:59.813638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:00:59.813638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:00:59.814998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:00:59.814998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:00:59.814998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:00:59.814998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:00:59.814998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:00:59.814998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:00:59.814998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:00:59.814998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:00:59.814998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:00:59.814998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:00:59.814998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 286/573] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Sep 6 16:00:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:00:59.814998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bef2b-3076-44bf-aecb-cdb72e39b674 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:59.848073 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca03056-3ef8-4a87-a214-bc399d7a3022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:59.848713 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca03056-3ef8-4a87-a214-bc399d7a3022 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:59.848989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca03056-3ef8-4a87-a214-bc399d7a3022 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:59.849394 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca03056-3ef8-4a87-a214-bc399d7a3022 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:59.850107 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca03056-3ef8-4a87-a214-bc399d7a3022 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:59.850416 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca03056-3ef8-4a87-a214-bc399d7a3022 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:59.851943 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca03056-3ef8-4a87-a214-bc399d7a3022 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:00:59.861723 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 288/574] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:00:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:00:59.863165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca03056-3ef8-4a87-a214-bc399d7a3022 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:00:59.943328 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bef2b-3076-44bf-aecb-cdb72e39b674 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:00:59.952142 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:59.986988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2000389-4294-4c4c-96d8-f19461ec2262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:00:59.987786 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2000389-4294-4c4c-96d8-f19461ec2262 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:00:59.988160 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2000389-4294-4c4c-96d8-f19461ec2262 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:00:59.988539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2000389-4294-4c4c-96d8-f19461ec2262 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:00:59.989408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:00:59.989559 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2000389-4294-4c4c-96d8-f19461ec2262 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:00:59.990451 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2000389-4294-4c4c-96d8-f19461ec2262 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:00:59.992910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2000389-4294-4c4c-96d8-f19461ec2262 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:00.009788 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 287/575] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:00:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:00.011376 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2000389-4294-4c4c-96d8-f19461ec2262 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:00.036319 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca03056-3ef8-4a87-a214-bc399d7a3022 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:00.091153 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdd62a-f0e6-47b8-a696-284579446938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:00.092296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdd62a-f0e6-47b8-a696-284579446938 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:00.092789 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdd62a-f0e6-47b8-a696-284579446938 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:00.093268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdd62a-f0e6-47b8-a696-284579446938 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:00.094613 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdd62a-f0e6-47b8-a696-284579446938 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:00.095182 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdd62a-f0e6-47b8-a696-284579446938 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:00.097878 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdd62a-f0e6-47b8-a696-284579446938 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:00.132026 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 289/576] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Sep 6 16:00:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:00.133336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2000389-4294-4c4c-96d8-f19461ec2262 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:00.133519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdd62a-f0e6-47b8-a696-284579446938 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:00.176293 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4323636-359a-4ce7-876a-021b58cb30ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:00.177003 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4323636-359a-4ce7-876a-021b58cb30ca None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:00.177929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4323636-359a-4ce7-876a-021b58cb30ca None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:00.178272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4323636-359a-4ce7-876a-021b58cb30ca None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:00.179339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4323636-359a-4ce7-876a-021b58cb30ca None admin] RBAC: Adding query filter params (name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:00.179835 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4323636-359a-4ce7-876a-021b58cb30ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:00.182140 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4323636-359a-4ce7-876a-021b58cb30ca None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:00.195033 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 288/577] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:01:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:00.195988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4323636-359a-4ce7-876a-021b58cb30ca None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:00.229876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:00.278977 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdd62a-f0e6-47b8-a696-284579446938 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:00.313741 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4323636-359a-4ce7-876a-021b58cb30ca None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:00.323901 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:00.343263 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacc46f-ade5-43c9-b0ed-1cbbec995796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:00.344290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacc46f-ade5-43c9-b0ed-1cbbec995796 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:00.344772 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacc46f-ade5-43c9-b0ed-1cbbec995796 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:00.345250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacc46f-ade5-43c9-b0ed-1cbbec995796 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:00.346583 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eacc46f-ade5-43c9-b0ed-1cbbec995796 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:00.347151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eacc46f-ade5-43c9-b0ed-1cbbec995796 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:00.349981 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eacc46f-ade5-43c9-b0ed-1cbbec995796 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:00.353757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d7516-6d7b-4207-b326-d40d07aa1ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:00.354538 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d7516-6d7b-4207-b326-d40d07aa1ecf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:00.355009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d7516-6d7b-4207-b326-d40d07aa1ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:00.355403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d7516-6d7b-4207-b326-d40d07aa1ecf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:00.367549 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d7516-6d7b-4207-b326-d40d07aa1ecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:00.369130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d7516-6d7b-4207-b326-d40d07aa1ecf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:00.379809 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 290/578] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Sep 6 16:01:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:00.381863 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacc46f-ade5-43c9-b0ed-1cbbec995796 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:00.391462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-e98d7516-6d7b-4207-b326-d40d07aa1ecf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:00.391462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:00.391462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:00.391462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:00.391462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:00.391462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:00.391462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:00.391462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:00.391462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:00.391462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:00.391462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:00.391462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:00.391462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:00.391462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:00.391462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:00.391462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:00.392638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:00.392638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:00.392638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:00.392638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:00.392638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:00.392638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:00.392638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:00.392638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:00.392638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:00.392638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:00.392638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:00.392638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:00.392638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:00.392638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:00.392638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:00.392638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:00.392638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:00.393733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:00.393733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:00.393733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:00.393939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 289/579] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:00.395201 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d7516-6d7b-4207-b326-d40d07aa1ecf None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:00.513764 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacc46f-ade5-43c9-b0ed-1cbbec995796 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:00.514030 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:00.537907 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d7516-6d7b-4207-b326-d40d07aa1ecf None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:00.554269 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4f505-3510-47f1-a876-bf478ba6a799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:00.554915 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4f505-3510-47f1-a876-bf478ba6a799 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:00.555194 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4f505-3510-47f1-a876-bf478ba6a799 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:00.555462 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4f505-3510-47f1-a876-bf478ba6a799 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:00.556260 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4f505-3510-47f1-a876-bf478ba6a799 None admin] RBAC: Adding query filter params (role.id=56d621cf28924933b6356795a6f8ba23, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=ece176d412574790a13d03dcca08f91a) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:00.556579 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4f505-3510-47f1-a876-bf478ba6a799 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:00.558021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4f505-3510-47f1-a876-bf478ba6a799 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:00.564376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 291/580] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:01:00 2022] GET /identity/v3/role_assignments?user.id=ece176d412574790a13d03dcca08f91a&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=56d621cf28924933b6356795a6f8ba23 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:00.565380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4f505-3510-47f1-a876-bf478ba6a799 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:00.585512 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955cac2c-e2d8-4b6e-9446-81d9508284f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:00.586598 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cac2c-e2d8-4b6e-9446-81d9508284f9 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:00.587116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cac2c-e2d8-4b6e-9446-81d9508284f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:00.587558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cac2c-e2d8-4b6e-9446-81d9508284f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:00.603730 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955cac2c-e2d8-4b6e-9446-81d9508284f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:00.607114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955cac2c-e2d8-4b6e-9446-81d9508284f9 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:00.626251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-955cac2c-e2d8-4b6e-9446-81d9508284f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:00.626251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:00.626251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:00.626251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:00.626251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:00.626251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:00.626251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:00.626251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:00.626251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:00.626251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:00.626251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:00.626251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:00.626251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:00.626251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:00.626251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:00.626251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:00.628431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:00.628431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:00.628431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:00.628431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:00.628431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:00.628431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:00.628431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:00.628431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:00.628431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:00.628431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:00.628431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:00.628431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:00.628431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:00.628431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:00.628431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:00.628431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:00.628431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:00.631104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:00.631104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:00.631104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:00.631104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 290/581] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:00.631104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955cac2c-e2d8-4b6e-9446-81d9508284f9 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:00.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:00.772699 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4f505-3510-47f1-a876-bf478ba6a799 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:00.784457 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955cac2c-e2d8-4b6e-9446-81d9508284f9 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:00.824618 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0304e-ea6e-44d8-9386-ce8481d91a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:00.825429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0304e-ea6e-44d8-9386-ce8481d91a49 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:00.825804 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0304e-ea6e-44d8-9386-ce8481d91a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:00.826183 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0304e-ea6e-44d8-9386-ce8481d91a49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:00.839971 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0304e-ea6e-44d8-9386-ce8481d91a49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:00.841996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:00.842179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0304e-ea6e-44d8-9386-ce8481d91a49 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:00.851001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e41b3-ad8f-4fce-af59-23498b4a23c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:00.852230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e41b3-ad8f-4fce-af59-23498b4a23c1 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:00.852839 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e41b3-ad8f-4fce-af59-23498b4a23c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:00.853430 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e41b3-ad8f-4fce-af59-23498b4a23c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:00.863769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-72c0304e-ea6e-44d8-9386-ce8481d91a49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:00.863769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:00.863769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:00.863769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:00.863769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:00.863769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:00.863769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:00.863769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:00.863769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:00.863769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:00.863769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:00.863769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:00.863769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:00.863769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:00.863769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:00.863769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:00.866375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:00.866375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:00.866375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:00.866375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:00.866375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:00.866375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:00.866375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:00.866375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:00.866375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:00.866375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:00.866375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:00.866375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:00.866375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:00.866375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:00.866375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:00.866375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:00.866375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:00.868324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:00.868324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:00.868324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:00.871701 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 292/582] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:00.873433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0304e-ea6e-44d8-9386-ce8481d91a49 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:00.878851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e41b3-ad8f-4fce-af59-23498b4a23c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:00.882329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e41b3-ad8f-4fce-af59-23498b4a23c1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:00.909854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-0c4e41b3-ad8f-4fce-af59-23498b4a23c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:00.909854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:00.909854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:00.909854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:00.909854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:00.909854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:00.909854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:00.909854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:00.909854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:00.909854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:00.909854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:00.909854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:00.909854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:00.909854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:00.909854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:00.909854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:00.911846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:00.911846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:00.911846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:00.911846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:00.911846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:00.911846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:00.911846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:00.911846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:00.911846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:00.911846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:00.911846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:00.911846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:00.911846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:00.911846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:00.911846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:00.911846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:00.911846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:00.913470 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:00.913470 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:00.913470 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:00.919579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 291/583] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:00.921375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e41b3-ad8f-4fce-af59-23498b4a23c1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:01.030005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:01.036323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0304e-ea6e-44d8-9386-ce8481d91a49 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:01.083171 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30600b-a0d2-4680-bbed-c3f5b242f1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:01.084773 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30600b-a0d2-4680-bbed-c3f5b242f1e3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:01.085264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30600b-a0d2-4680-bbed-c3f5b242f1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:01.085719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30600b-a0d2-4680-bbed-c3f5b242f1e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:01.086832 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30600b-a0d2-4680-bbed-c3f5b242f1e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:01.087325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30600b-a0d2-4680-bbed-c3f5b242f1e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:01.089737 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30600b-a0d2-4680-bbed-c3f5b242f1e3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:01.111020 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 293/584] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:01.112852 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30600b-a0d2-4680-bbed-c3f5b242f1e3 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:01.134873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e41b3-ad8f-4fce-af59-23498b4a23c1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:01.180916 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53869fc0-de1e-4169-b824-3477810d6f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:01.181806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53869fc0-de1e-4169-b824-3477810d6f93 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:01.182238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53869fc0-de1e-4169-b824-3477810d6f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:01.182700 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53869fc0-de1e-4169-b824-3477810d6f93 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:01.183531 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53869fc0-de1e-4169-b824-3477810d6f93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:01.183991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53869fc0-de1e-4169-b824-3477810d6f93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:01.185536 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53869fc0-de1e-4169-b824-3477810d6f93 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:01.206981 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 292/585] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:01.208749 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53869fc0-de1e-4169-b824-3477810d6f93 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:01.273381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:01.312407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30600b-a0d2-4680-bbed-c3f5b242f1e3 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:01.358096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53869fc0-de1e-4169-b824-3477810d6f93 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:01.379042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb299c-a71a-4df1-b5c7-6dfd42e49634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:01.380794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb299c-a71a-4df1-b5c7-6dfd42e49634 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:01.381190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb299c-a71a-4df1-b5c7-6dfd42e49634 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:01.381684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb299c-a71a-4df1-b5c7-6dfd42e49634 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:01.382871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb299c-a71a-4df1-b5c7-6dfd42e49634 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:01.383412 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb299c-a71a-4df1-b5c7-6dfd42e49634 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:01.385981 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb299c-a71a-4df1-b5c7-6dfd42e49634 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:01.413142 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 294/586] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:01.413142 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb299c-a71a-4df1-b5c7-6dfd42e49634 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:01.432054 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eaaa88-a0ae-4236-8b70-b06d35ca0378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:01.433270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eaaa88-a0ae-4236-8b70-b06d35ca0378 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:01.433541 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eaaa88-a0ae-4236-8b70-b06d35ca0378 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:01.434032 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eaaa88-a0ae-4236-8b70-b06d35ca0378 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:01.435221 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eaaa88-a0ae-4236-8b70-b06d35ca0378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:01.435748 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eaaa88-a0ae-4236-8b70-b06d35ca0378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:01.438363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eaaa88-a0ae-4236-8b70-b06d35ca0378 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:01.446491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:01.462726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 293/587] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:01.464255 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eaaa88-a0ae-4236-8b70-b06d35ca0378 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:01.590638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb299c-a71a-4df1-b5c7-6dfd42e49634 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:01.635217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63207ea8-648b-44c4-bf53-b41784a35069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:01.635893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63207ea8-648b-44c4-bf53-b41784a35069 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:01.636177 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63207ea8-648b-44c4-bf53-b41784a35069 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:01.636440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63207ea8-648b-44c4-bf53-b41784a35069 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:01.651031 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63207ea8-648b-44c4-bf53-b41784a35069 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:01.653816 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63207ea8-648b-44c4-bf53-b41784a35069 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:01.678296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-63207ea8-648b-44c4-bf53-b41784a35069 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 16:01:01.678296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:01.678296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:01.678296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:01.678296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:01.678296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:01.678296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:01.678296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:01.678296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:01.678296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:01.678296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:01.678296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:01.678296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:01.678296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:01.678296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:01.678296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:01.679362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:01.679362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:01.679362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:01.679362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:01.679362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:01.679362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:01.679362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:01.679362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:01.679362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:01.679362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:01.679362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:01.679362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:01.679362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:01.679362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:01.679362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:01.679362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:01.679362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:01.679362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:01.680286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:01.680286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:01.680286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:01.680286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:01.680286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:01.680286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:01.680286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:01.680286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:01.680286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 16:01:01.680286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:01.681253 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eaaa88-a0ae-4236-8b70-b06d35ca0378 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:01.682986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 295/588] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:01:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:01.685077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:01.685797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63207ea8-648b-44c4-bf53-b41784a35069 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:01.747833 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cd5e5-1b10-46b6-983d-08836fdf47af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:01.748918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cd5e5-1b10-46b6-983d-08836fdf47af None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:01.749403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cd5e5-1b10-46b6-983d-08836fdf47af None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:01.749864 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cd5e5-1b10-46b6-983d-08836fdf47af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:01.762907 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195cd5e5-1b10-46b6-983d-08836fdf47af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:01.765692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195cd5e5-1b10-46b6-983d-08836fdf47af None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:01.788373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-195cd5e5-1b10-46b6-983d-08836fdf47af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:01.788373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:01.788373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:01.788373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:01.788373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:01.788373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:01.788373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:01.788373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:01.788373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:01.788373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:01.788373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:01.788373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:01.788373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:01.788373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:01.788373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:01.790005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:01.790005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:01.790005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:01.790005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:01.790005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:01.790005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:01.790005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:01.790005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:01.790005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:01.790005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:01.790005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:01.790005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:01.790005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:01.790005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:01.790005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:01.790005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:01.790005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:01.791102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:01.791102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:01.791102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:01.791102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:01.791102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:01.791102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:01.791102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:01.791102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:01.791525 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 294/589] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:01.792911 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cd5e5-1b10-46b6-983d-08836fdf47af None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:01.810386 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:01.850269 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63207ea8-648b-44c4-bf53-b41784a35069 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:01.907617 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7a974-87cd-4e2d-83f2-af0bc4024131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:01.908727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7a974-87cd-4e2d-83f2-af0bc4024131 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:01.909220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7a974-87cd-4e2d-83f2-af0bc4024131 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:01.909893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7a974-87cd-4e2d-83f2-af0bc4024131 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:01.925394 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7a974-87cd-4e2d-83f2-af0bc4024131 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:01.928632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7a974-87cd-4e2d-83f2-af0bc4024131 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:01.951638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-46e7a974-87cd-4e2d-83f2-af0bc4024131 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 16:01:01.951638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:01.951638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:01.951638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:01.951638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:01.951638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:01.951638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:01.951638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:01.951638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:01.951638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:01.951638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:01.951638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:01.951638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:01.951638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:01.951638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:01.951638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:01.952638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:01.952638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:01.952638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:01.952638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:01.952638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:01.952638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:01.952638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:01.952638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:01.952638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:01.952638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:01.952638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:01.952638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:01.952638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:01.952638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:01.952638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:01.952638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:01.952638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:01.952638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:01.953589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:01.953589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:01.953589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:01.953589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:01.953589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:01.953589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:01.953589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:01.953589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:01.953589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 16:01:01.953589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:01.954128 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 296/590] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:01:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:01.955033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7a974-87cd-4e2d-83f2-af0bc4024131 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:01.968961 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cd5e5-1b10-46b6-983d-08836fdf47af None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:02.024455 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def94683-45ea-4f22-b4b2-b9db790d7dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:02.025106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def94683-45ea-4f22-b4b2-b9db790d7dfe None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:02.025386 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def94683-45ea-4f22-b4b2-b9db790d7dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:02.025659 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def94683-45ea-4f22-b4b2-b9db790d7dfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:02.038443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def94683-45ea-4f22-b4b2-b9db790d7dfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:02.044467 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def94683-45ea-4f22-b4b2-b9db790d7dfe None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:02.064184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-def94683-45ea-4f22-b4b2-b9db790d7dfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:02.064184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:02.064184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:02.064184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:02.064184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:02.064184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:02.064184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:02.064184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:02.064184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:02.064184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:02.064184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:02.064184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:02.064184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:02.064184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:02.064184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:02.065138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:02.065138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:02.065138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:02.065138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:02.065138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:02.065138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:02.065138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:02.065138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:02.065138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:02.065138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:02.065138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:02.065138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:02.065138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:02.065138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:02.065138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:02.065138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:02.065138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:02.066016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:02.066016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:02.066016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:02.066016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:02.066016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:02.066016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:02.066016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:02.066016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:02.066016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 295/591] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:02.066495 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def94683-45ea-4f22-b4b2-b9db790d7dfe None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:02.105547 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:02.152661 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7a974-87cd-4e2d-83f2-af0bc4024131 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:02.210057 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d088933e-ce25-4abc-a226-e5a4c6d45f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:02.210988 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d088933e-ce25-4abc-a226-e5a4c6d45f2a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:02.211342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d088933e-ce25-4abc-a226-e5a4c6d45f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:02.211675 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d088933e-ce25-4abc-a226-e5a4c6d45f2a None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:02.212532 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d088933e-ce25-4abc-a226-e5a4c6d45f2a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:02.212916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d088933e-ce25-4abc-a226-e5a4c6d45f2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:02.214534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d088933e-ce25-4abc-a226-e5a4c6d45f2a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:02.215023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def94683-45ea-4f22-b4b2-b9db790d7dfe None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:02.225731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:02.242997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 297/592] 10.222.0.26 () {56 vars in 1139 bytes} [Tue Sep 6 16:01:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:02.243768 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d088933e-ce25-4abc-a226-e5a4c6d45f2a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:02.282972 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26696e-d62f-473e-ab4f-f773c8b8620c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:02.283620 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26696e-d62f-473e-ab4f-f773c8b8620c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:02.283933 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26696e-d62f-473e-ab4f-f773c8b8620c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:02.284211 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26696e-d62f-473e-ab4f-f773c8b8620c None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:02.284923 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26696e-d62f-473e-ab4f-f773c8b8620c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:02.285229 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26696e-d62f-473e-ab4f-f773c8b8620c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:02.287201 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26696e-d62f-473e-ab4f-f773c8b8620c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:02.311858 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 296/593] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:02.314024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26696e-d62f-473e-ab4f-f773c8b8620c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:02.428028 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d088933e-ce25-4abc-a226-e5a4c6d45f2a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:02.458997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:02.470055 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10982a-8033-463a-8787-e9b09d827518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:02.471106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10982a-8033-463a-8787-e9b09d827518 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:02.471393 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10982a-8033-463a-8787-e9b09d827518 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:02.471731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10982a-8033-463a-8787-e9b09d827518 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:02.472422 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10982a-8033-463a-8787-e9b09d827518 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:02.472907 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10982a-8033-463a-8787-e9b09d827518 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:02.474443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10982a-8033-463a-8787-e9b09d827518 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:02.491798 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26696e-d62f-473e-ab4f-f773c8b8620c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:02.495223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 298/594] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Sep 6 16:01:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:02.495945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10982a-8033-463a-8787-e9b09d827518 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:02.557790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40573c7-ddf1-4cf7-b3e3-f04a70488981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:02.558408 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40573c7-ddf1-4cf7-b3e3-f04a70488981 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:02.559001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40573c7-ddf1-4cf7-b3e3-f04a70488981 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:02.559282 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40573c7-ddf1-4cf7-b3e3-f04a70488981 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:02.559969 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40573c7-ddf1-4cf7-b3e3-f04a70488981 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:02.560832 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40573c7-ddf1-4cf7-b3e3-f04a70488981 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:02.565972 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40573c7-ddf1-4cf7-b3e3-f04a70488981 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:02.576722 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:02.581370 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 297/595] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:02.582807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40573c7-ddf1-4cf7-b3e3-f04a70488981 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:02.662540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10982a-8033-463a-8787-e9b09d827518 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:02.732618 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c149a9-14ac-439a-8f61-3e9f299e008e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:02.733674 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c149a9-14ac-439a-8f61-3e9f299e008e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:02.733674 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c149a9-14ac-439a-8f61-3e9f299e008e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:02.733960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c149a9-14ac-439a-8f61-3e9f299e008e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:02.743669 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40573c7-ddf1-4cf7-b3e3-f04a70488981 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:02.751970 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c149a9-14ac-439a-8f61-3e9f299e008e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:02.754456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c149a9-14ac-439a-8f61-3e9f299e008e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:02.780702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-36c149a9-14ac-439a-8f61-3e9f299e008e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:02.780702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:02.780702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:02.780702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:02.780702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:02.780702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:02.780702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:02.780702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:02.780702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:02.780702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:02.780702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:02.780702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:02.780702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:02.780702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:02.780702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:02.780702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:02.781730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:02.781730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:02.781730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:02.781730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:02.781730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:02.781730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:02.781730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:02.781730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:02.781730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:02.781730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:02.781730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:02.781730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:02.781730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:02.781730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:02.781730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:02.781730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:02.781730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:02.782620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:02.782620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:02.782620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:02.782620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 299/596] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:02.783940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c149a9-14ac-439a-8f61-3e9f299e008e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:02.805966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41635e94-dd80-4536-843a-a25d66ef2763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:02.807055 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41635e94-dd80-4536-843a-a25d66ef2763 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:02.807503 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41635e94-dd80-4536-843a-a25d66ef2763 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:02.807957 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41635e94-dd80-4536-843a-a25d66ef2763 None admin] PATH_INFO: `/v3/projects/0138d62d5879495499a17e20310d4f61/users/e50ef7db27744c7ebd2ec95cecc9d1a8/roles/56d621cf28924933b6356795a6f8ba23` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:02.833681 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41635e94-dd80-4536-843a-a25d66ef2763 None admin] RBAC: Authorizing `identity:create_grant(project_id=0138d62d5879495499a17e20310d4f61, user_id=e50ef7db27744c7ebd2ec95cecc9d1a8, role_id=56d621cf28924933b6356795a6f8ba23)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:02.836396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41635e94-dd80-4536-843a-a25d66ef2763 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:02.840055 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:02.857263 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 298/597] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:01:02 2022] PUT /identity/v3/projects/0138d62d5879495499a17e20310d4f61/users/e50ef7db27744c7ebd2ec95cecc9d1a8/roles/56d621cf28924933b6356795a6f8ba23 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:01:02.858715 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41635e94-dd80-4536-843a-a25d66ef2763 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:02.956204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c149a9-14ac-439a-8f61-3e9f299e008e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:02.967324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:03.008587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77789f5d-55ce-4a67-8bcb-c91935eb777c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:03.009370 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77789f5d-55ce-4a67-8bcb-c91935eb777c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:03.009863 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77789f5d-55ce-4a67-8bcb-c91935eb777c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:03.010457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77789f5d-55ce-4a67-8bcb-c91935eb777c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:03.025234 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77789f5d-55ce-4a67-8bcb-c91935eb777c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:03.027564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77789f5d-55ce-4a67-8bcb-c91935eb777c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:03.043121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41635e94-dd80-4536-843a-a25d66ef2763 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:03.047265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-77789f5d-55ce-4a67-8bcb-c91935eb777c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:03.047265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:03.047265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:03.047265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:03.047265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:03.047265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:03.047265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:03.047265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:03.047265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:03.047265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:03.047265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:03.047265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:03.047265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:03.047265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:03.047265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:03.047265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:03.049362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:03.049362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:03.049362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:03.049362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:03.049362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:03.049362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:03.049362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:03.049362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:03.049362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:03.049362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:03.049362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:03.049362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:03.049362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:03.049362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:03.049362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:03.049362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:03.049362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:03.051724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:03.051724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:03.051724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:03.051724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 300/598] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:03.051724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8daa8-c1bc-4163-b5c8-c609cdcbc28e None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:03.052856 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8daa8-c1bc-4163-b5c8-c609cdcbc28e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:03.053629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8daa8-c1bc-4163-b5c8-c609cdcbc28e None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:03.055785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 301/599] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:03.056666 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8daa8-c1bc-4163-b5c8-c609cdcbc28e None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:03.115997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52a5d2-193d-4bcf-be95-c058e40dbe19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:03.116944 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52a5d2-193d-4bcf-be95-c058e40dbe19 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:03.117400 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52a5d2-193d-4bcf-be95-c058e40dbe19 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:03.117840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52a5d2-193d-4bcf-be95-c058e40dbe19 None admin] PATH_INFO: `/v3/projects/0138d62d5879495499a17e20310d4f61/users/6afd4f6ffdae4b16aa5ba757e68c18c4/roles/56d621cf28924933b6356795a6f8ba23` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:03.137377 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52a5d2-193d-4bcf-be95-c058e40dbe19 None admin] RBAC: Authorizing `identity:create_grant(project_id=0138d62d5879495499a17e20310d4f61, user_id=6afd4f6ffdae4b16aa5ba757e68c18c4, role_id=56d621cf28924933b6356795a6f8ba23)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:03.139777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52a5d2-193d-4bcf-be95-c058e40dbe19 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:03.157426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 299/600] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:01:02 2022] PUT /identity/v3/projects/0138d62d5879495499a17e20310d4f61/users/6afd4f6ffdae4b16aa5ba757e68c18c4/roles/56d621cf28924933b6356795a6f8ba23 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:01:03.159311 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa1c0a-0e48-4621-a64a-99f7d020b6cc None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:03.159858 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa1c0a-0e48-4621-a64a-99f7d020b6cc None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:03.160336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa1c0a-0e48-4621-a64a-99f7d020b6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:03.182274 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-5afa1c0a-0e48-4621-a64a-99f7d020b6cc None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:03.183920 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5afa1c0a-0e48-4621-a64a-99f7d020b6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:03.247385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8daa8-c1bc-4163-b5c8-c609cdcbc28e None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:03.258162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:03.294748 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e24dc0-bce9-48e1-9a0b-592728c6bc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:03.295749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e24dc0-bce9-48e1-9a0b-592728c6bc30 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:03.296360 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e24dc0-bce9-48e1-9a0b-592728c6bc30 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:03.296891 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e24dc0-bce9-48e1-9a0b-592728c6bc30 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:03.297951 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e24dc0-bce9-48e1-9a0b-592728c6bc30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:03.298521 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e24dc0-bce9-48e1-9a0b-592728c6bc30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:03.300613 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e24dc0-bce9-48e1-9a0b-592728c6bc30 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:03.317932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 300/601] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:01:03.317932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa1c0a-0e48-4621-a64a-99f7d020b6cc None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:03.331151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 302/602] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:03.337324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e24dc0-bce9-48e1-9a0b-592728c6bc30 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:03.373193 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:03.459277 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa1c0a-0e48-4621-a64a-99f7d020b6cc None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:03.515009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd03a49-12c1-4895-b6d9-e23a5c5334cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:03.516469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd03a49-12c1-4895-b6d9-e23a5c5334cc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:03.517370 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd03a49-12c1-4895-b6d9-e23a5c5334cc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:03.518211 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd03a49-12c1-4895-b6d9-e23a5c5334cc None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:03.519646 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd03a49-12c1-4895-b6d9-e23a5c5334cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:03.520464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd03a49-12c1-4895-b6d9-e23a5c5334cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:03.523682 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd03a49-12c1-4895-b6d9-e23a5c5334cc None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:03.536117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e24dc0-bce9-48e1-9a0b-592728c6bc30 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:03.552073 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 301/603] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:03.561794 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd03a49-12c1-4895-b6d9-e23a5c5334cc None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:03.587223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132af71-4552-4594-b095-82f246ce99bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:03.589633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132af71-4552-4594-b095-82f246ce99bc None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:03.589950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132af71-4552-4594-b095-82f246ce99bc None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:03.590194 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132af71-4552-4594-b095-82f246ce99bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:03.603755 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132af71-4552-4594-b095-82f246ce99bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:03.605258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132af71-4552-4594-b095-82f246ce99bc None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:03.637552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-1132af71-4552-4594-b095-82f246ce99bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:03.637552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:03.637552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:03.637552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:03.637552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:03.637552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:03.637552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:03.637552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:03.637552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:03.637552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:03.637552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:03.637552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:03.637552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:03.637552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:03.637552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:03.639261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:03.639261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:03.639261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:03.639261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:03.639261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:03.639261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:03.639261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:03.639261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:03.639261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:03.639261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:03.639261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:03.639261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:03.639261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:03.639261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:03.639261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:03.639261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:03.639261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:03.641015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:03.641015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:03.641015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:03.641015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:03.641015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:03.641015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:03.641015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:03.641015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:03.641865 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:03.647035 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 303/604] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:03.648228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132af71-4552-4594-b095-82f246ce99bc None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:03.784952 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd03a49-12c1-4895-b6d9-e23a5c5334cc None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:03.830475 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:03.852230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67132026-bbfc-4f2f-8a24-2c3a5220cf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:03.852873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67132026-bbfc-4f2f-8a24-2c3a5220cf30 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:03.853128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67132026-bbfc-4f2f-8a24-2c3a5220cf30 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:03.853394 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67132026-bbfc-4f2f-8a24-2c3a5220cf30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:03.865749 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67132026-bbfc-4f2f-8a24-2c3a5220cf30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:03.867330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67132026-bbfc-4f2f-8a24-2c3a5220cf30 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:03.875890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132af71-4552-4594-b095-82f246ce99bc None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:03.883664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-67132026-bbfc-4f2f-8a24-2c3a5220cf30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:03.883664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:03.883664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:03.883664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:03.883664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:03.883664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:03.883664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:03.883664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:03.883664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:03.883664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:03.883664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:03.883664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:03.883664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:03.883664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:03.883664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:03.884742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:03.884742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:03.884742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:03.884742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:03.884742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:03.884742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:03.884742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:03.884742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:03.884742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:03.884742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:03.884742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:03.884742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:03.884742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:03.884742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:03.884742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:03.884742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:03.884742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:03.885837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:03.885837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:03.885837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:03.885837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:03.885837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:03.885837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:03.885837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:03.885837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:03.885837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 302/605] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:03.886332 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67132026-bbfc-4f2f-8a24-2c3a5220cf30 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:03.930599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f255a15-de36-413b-8c23-0a8afd36cc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:03.931720 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f255a15-de36-413b-8c23-0a8afd36cc02 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:03.932162 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f255a15-de36-413b-8c23-0a8afd36cc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:03.932984 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f255a15-de36-413b-8c23-0a8afd36cc02 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:03.934144 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f255a15-de36-413b-8c23-0a8afd36cc02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:03.934614 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f255a15-de36-413b-8c23-0a8afd36cc02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:03.937258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f255a15-de36-413b-8c23-0a8afd36cc02 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:03.955254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 304/606] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:03.956824 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f255a15-de36-413b-8c23-0a8afd36cc02 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:04.030018 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:04.040839 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67132026-bbfc-4f2f-8a24-2c3a5220cf30 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:04.087864 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f2a02-0fdd-439e-a40c-5bc93c2af6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:04.088434 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f2a02-0fdd-439e-a40c-5bc93c2af6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:04.088712 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f2a02-0fdd-439e-a40c-5bc93c2af6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:04.089005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f2a02-0fdd-439e-a40c-5bc93c2af6a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:04.096805 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f255a15-de36-413b-8c23-0a8afd36cc02 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:04.104909 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f2a02-0fdd-439e-a40c-5bc93c2af6a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:01:04.111678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-373f2a02-0fdd-439e-a40c-5bc93c2af6a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:01:04.111678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:04.111678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:04.111678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:04.111678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:04.111678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:04.111678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:04.111678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:04.111678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:04.111678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:04.111678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:04.111678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:04.111678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:01:04.111678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:01:04.111678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:01:04.111678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:01:04.112747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:01:04.112747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:01:04.112747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:01:04.112747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:01:04.112747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:04.112747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:04.112747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:04.112747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:04.112747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:04.112747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:04.112747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:04.112747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:04.112747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:04.112747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:04.112747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:04.112747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:04.118790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:04.118790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:04.118790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:04.118790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:04.118790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:01:04.118790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:01:04.118790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:01:04.118790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:01:04.118790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:01:04.118790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:01:04.118790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:01:04.118790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:04.118790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 303/607] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:01:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:04.118790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f2a02-0fdd-439e-a40c-5bc93c2af6a7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:04.160761 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d227a33-71f7-44fb-b511-978a10c8a243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:04.162260 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d227a33-71f7-44fb-b511-978a10c8a243 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:04.162724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d227a33-71f7-44fb-b511-978a10c8a243 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:04.163144 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d227a33-71f7-44fb-b511-978a10c8a243 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:04.164473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d227a33-71f7-44fb-b511-978a10c8a243 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:04.166085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d227a33-71f7-44fb-b511-978a10c8a243 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:04.167786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d227a33-71f7-44fb-b511-978a10c8a243 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:04.178838 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:04.191039 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 305/608] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:04.192470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d227a33-71f7-44fb-b511-978a10c8a243 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:04.245863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f2a02-0fdd-439e-a40c-5bc93c2af6a7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:04.285493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219205d0-bc15-4392-a752-901e01aa4cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:04.286169 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219205d0-bc15-4392-a752-901e01aa4cc2 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:04.287001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219205d0-bc15-4392-a752-901e01aa4cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:04.287745 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219205d0-bc15-4392-a752-901e01aa4cc2 None admin] PATH_INFO: `/v3/projects/0138d62d5879495499a17e20310d4f61/users/989beaca146d4f9b950f333bd3567e6a/roles/56d621cf28924933b6356795a6f8ba23` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:04.306376 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219205d0-bc15-4392-a752-901e01aa4cc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=0138d62d5879495499a17e20310d4f61, user_id=989beaca146d4f9b950f333bd3567e6a, role_id=56d621cf28924933b6356795a6f8ba23)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:04.307891 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219205d0-bc15-4392-a752-901e01aa4cc2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:04.313008 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d227a33-71f7-44fb-b511-978a10c8a243 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:04.327086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 304/609] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:01:04 2022] PUT /identity/v3/projects/0138d62d5879495499a17e20310d4f61/users/989beaca146d4f9b950f333bd3567e6a/roles/56d621cf28924933b6356795a6f8ba23 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:01:04.328458 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219205d0-bc15-4392-a752-901e01aa4cc2 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:04.333790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:04.356374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993d8e4-8db7-4f7e-8861-419143c45321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:04.357286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993d8e4-8db7-4f7e-8861-419143c45321 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:04.357499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993d8e4-8db7-4f7e-8861-419143c45321 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:04.357888 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993d8e4-8db7-4f7e-8861-419143c45321 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:04.358734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4993d8e4-8db7-4f7e-8861-419143c45321 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:04.359185 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4993d8e4-8db7-4f7e-8861-419143c45321 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:04.360727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4993d8e4-8db7-4f7e-8861-419143c45321 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:04.375183 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 306/610] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:01:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:04.378095 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993d8e4-8db7-4f7e-8861-419143c45321 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:04.517113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219205d0-bc15-4392-a752-901e01aa4cc2 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:04.542626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993d8e4-8db7-4f7e-8861-419143c45321 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:04.547406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:04.603626 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b0e6d-2032-442d-9919-89c413942995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:04.604321 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b0e6d-2032-442d-9919-89c413942995 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:04.604602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b0e6d-2032-442d-9919-89c413942995 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:04.604876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b0e6d-2032-442d-9919-89c413942995 None admin] PATH_INFO: `/v3/projects/0138d62d5879495499a17e20310d4f61/users/7849ef049ffd4495909f997735af8782/roles/56d621cf28924933b6356795a6f8ba23` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:04.623078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46801e68-5bfb-4240-8d4b-bf3f388323fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:04.624152 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46801e68-5bfb-4240-8d4b-bf3f388323fa None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:04.624783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46801e68-5bfb-4240-8d4b-bf3f388323fa None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:04.625473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46801e68-5bfb-4240-8d4b-bf3f388323fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:04.626697 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b0e6d-2032-442d-9919-89c413942995 None admin] RBAC: Authorizing `identity:create_grant(project_id=0138d62d5879495499a17e20310d4f61, user_id=7849ef049ffd4495909f997735af8782, role_id=56d621cf28924933b6356795a6f8ba23)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:04.628195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b0e6d-2032-442d-9919-89c413942995 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:04.649515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 305/611] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:01:04 2022] PUT /identity/v3/projects/0138d62d5879495499a17e20310d4f61/users/7849ef049ffd4495909f997735af8782/roles/56d621cf28924933b6356795a6f8ba23 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:01:04.655743 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46801e68-5bfb-4240-8d4b-bf3f388323fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:04.657805 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46801e68-5bfb-4240-8d4b-bf3f388323fa None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:04.680738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-46801e68-5bfb-4240-8d4b-bf3f388323fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:04.680738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:04.680738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:04.680738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:04.680738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:04.680738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:04.680738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:04.680738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:04.680738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:04.680738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:04.680738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:04.680738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:04.680738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:04.680738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:04.680738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:04.680738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:04.681724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:04.681724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:04.681724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:04.681724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:04.681724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:04.681724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:04.681724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:04.681724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:04.681724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:04.681724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:04.681724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:04.681724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:04.681724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:04.681724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:04.681724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:04.681724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:04.681724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:04.683037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:04.683037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:04.683037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:04.683037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 307/612] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:04.691189 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b0e6d-2032-442d-9919-89c413942995 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:04.787855 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:04.878077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b0e6d-2032-442d-9919-89c413942995 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:04.947799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1f15b-fbb1-4d16-9219-45e37afbb333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:04.948833 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1f15b-fbb1-4d16-9219-45e37afbb333 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:04.949320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1f15b-fbb1-4d16-9219-45e37afbb333 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:04.949791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1f15b-fbb1-4d16-9219-45e37afbb333 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:04.950997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1f15b-fbb1-4d16-9219-45e37afbb333 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:04.951477 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1f15b-fbb1-4d16-9219-45e37afbb333 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:04.954162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1f15b-fbb1-4d16-9219-45e37afbb333 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:04.979395 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 306/613] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:04.985177 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46801e68-5bfb-4240-8d4b-bf3f388323fa None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:05.180142 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46801e68-5bfb-4240-8d4b-bf3f388323fa None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:05.215665 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:05.269455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e3d0a-a8c5-4cc7-8aaa-7c4bb26d547e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:05.270581 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e3d0a-a8c5-4cc7-8aaa-7c4bb26d547e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:05.272283 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e3d0a-a8c5-4cc7-8aaa-7c4bb26d547e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:05.272787 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e3d0a-a8c5-4cc7-8aaa-7c4bb26d547e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:05.295476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e3d0a-a8c5-4cc7-8aaa-7c4bb26d547e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:05.298217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e3d0a-a8c5-4cc7-8aaa-7c4bb26d547e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:05.320994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-ba8e3d0a-a8c5-4cc7-8aaa-7c4bb26d547e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:01:05.320994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:05.320994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:05.320994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:05.320994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:05.320994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:05.320994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:05.320994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:05.320994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:05.320994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:05.320994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:05.320994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:05.320994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:05.320994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:05.320994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:05.320994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:05.321997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:05.321997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:05.321997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:05.321997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:05.321997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:05.321997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:05.321997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:05.321997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:05.321997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:05.321997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:05.321997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:05.321997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:05.321997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:05.321997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:05.321997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:05.321997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:05.321997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:05.321997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:05.323382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:05.323382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:05.323382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:05.323382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:05.323382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:05.323382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:05.323382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:05.323382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:05.323382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:01:05.323382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:05.326749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 308/614] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:01:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:05.336854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1f15b-fbb1-4d16-9219-45e37afbb333 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:05.459169 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4d413-887c-4ea6-b295-ce9834b45230 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:05.460031 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4d413-887c-4ea6-b295-ce9834b45230 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:05.460715 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4d413-887c-4ea6-b295-ce9834b45230 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:05.468911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 309/615] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:05.471740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa972e2-117e-4c85-89f5-bf1762ac628d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:05.472457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa972e2-117e-4c85-89f5-bf1762ac628d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:05.473135 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa972e2-117e-4c85-89f5-bf1762ac628d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:05.506415 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-dfa972e2-117e-4c85-89f5-bf1762ac628d None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:05.508059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-dfa972e2-117e-4c85-89f5-bf1762ac628d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:05.639072 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 310/616] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:01:05.651330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1f15b-fbb1-4d16-9219-45e37afbb333 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:05.675478 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:05.705160 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963c988-d666-4431-b31f-f0b29d970b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:05.705781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963c988-d666-4431-b31f-f0b29d970b7c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:05.706069 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963c988-d666-4431-b31f-f0b29d970b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:05.706321 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963c988-d666-4431-b31f-f0b29d970b7c None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:05.707107 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3963c988-d666-4431-b31f-f0b29d970b7c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:05.707418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3963c988-d666-4431-b31f-f0b29d970b7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:05.709547 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3963c988-d666-4431-b31f-f0b29d970b7c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:05.741475 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 307/617] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Sep 6 16:01:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:05.747727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa972e2-117e-4c85-89f5-bf1762ac628d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:05.998487 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa972e2-117e-4c85-89f5-bf1762ac628d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:06.001148 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:06.098255 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130ce86-19e2-417d-a32a-34ec22311a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:06.099881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130ce86-19e2-417d-a32a-34ec22311a70 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:06.100349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130ce86-19e2-417d-a32a-34ec22311a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:06.100831 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130ce86-19e2-417d-a32a-34ec22311a70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:06.124833 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130ce86-19e2-417d-a32a-34ec22311a70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:06.127502 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130ce86-19e2-417d-a32a-34ec22311a70 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:06.159606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-5130ce86-19e2-417d-a32a-34ec22311a70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:06.159606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:06.159606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:06.159606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:06.159606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:06.159606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:06.159606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:06.159606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:06.159606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:06.159606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:06.159606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:06.159606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:06.159606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:06.159606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:06.159606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:06.159606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:06.161485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:06.161485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:06.161485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:06.161485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:06.161485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:06.161485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:06.161485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:06.161485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:06.161485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:06.161485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:06.161485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:06.161485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:06.161485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:06.161485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:06.161485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:06.161485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:06.161485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:06.164758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:06.164758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:06.164758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:06.164758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 311/618] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:06.168824 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963c988-d666-4431-b31f-f0b29d970b7c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:06.188316 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130ce86-19e2-417d-a32a-34ec22311a70 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:06.435362 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963c988-d666-4431-b31f-f0b29d970b7c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:06.451893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130ce86-19e2-417d-a32a-34ec22311a70 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:06.482435 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3d008-cde4-45de-b6bb-4dcc62328e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:06.483207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3d008-cde4-45de-b6bb-4dcc62328e0c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:06.483480 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3d008-cde4-45de-b6bb-4dcc62328e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:06.483756 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3d008-cde4-45de-b6bb-4dcc62328e0c None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:06.484518 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3d008-cde4-45de-b6bb-4dcc62328e0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:06.484827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3d008-cde4-45de-b6bb-4dcc62328e0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:06.486223 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3d008-cde4-45de-b6bb-4dcc62328e0c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:06.500468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 308/619] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:06.501891 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f42bc-a126-4345-81c0-f8a4ecd9a46f None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:06.502181 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f42bc-a126-4345-81c0-f8a4ecd9a46f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:06.502455 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f42bc-a126-4345-81c0-f8a4ecd9a46f None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:06.504054 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 309/620] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:06.511433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f42bc-a126-4345-81c0-f8a4ecd9a46f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:06.514175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f3140-0be0-4a65-82a5-4a1ea9c39c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:06.514873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f3140-0be0-4a65-82a5-4a1ea9c39c1a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:06.515158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f3140-0be0-4a65-82a5-4a1ea9c39c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:06.515437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f3140-0be0-4a65-82a5-4a1ea9c39c1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:06.528622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f3140-0be0-4a65-82a5-4a1ea9c39c1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:01:06.535431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-fe3f3140-0be0-4a65-82a5-4a1ea9c39c1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:01:06.535431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:06.535431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:06.535431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:06.535431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:06.535431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:06.535431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:06.535431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:06.535431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:06.535431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:06.535431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:06.535431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:06.535431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:01:06.535431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:01:06.535431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:01:06.535431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:01:06.537009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:01:06.537009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:01:06.537009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:01:06.537009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:01:06.537009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:06.537009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:06.537009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:06.537009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:06.537009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:06.537009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:06.537009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:06.537009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:06.537009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:06.537009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:06.537009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:06.537009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:06.537828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:06.537828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:06.537828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:06.537828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:06.537828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:01:06.537828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:01:06.537828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:01:06.537828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:01:06.537828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:01:06.537828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:01:06.537828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:01:06.537828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:06.538636 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 312/621] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:01:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:06.540710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db79bf-1d4a-4e57-a907-95aeb0d18011 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:06.541108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db79bf-1d4a-4e57-a907-95aeb0d18011 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:06.541475 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db79bf-1d4a-4e57-a907-95aeb0d18011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:06.566653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:06.575302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-11db79bf-1d4a-4e57-a907-95aeb0d18011 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:06.577473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-11db79bf-1d4a-4e57-a907-95aeb0d18011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:06.596198 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:06.658931 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f42bc-a126-4345-81c0-f8a4ecd9a46f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:06.700926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6e5ce-5feb-469d-9662-423f993ea145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:06.701639 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6e5ce-5feb-469d-9662-423f993ea145 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:06.701900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6e5ce-5feb-469d-9662-423f993ea145 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:06.702139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6e5ce-5feb-469d-9662-423f993ea145 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:06.710744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6e5ce-5feb-469d-9662-423f993ea145 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:06.712187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6e5ce-5feb-469d-9662-423f993ea145 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:06.721112 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 313/622] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:01:06.723043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db79bf-1d4a-4e57-a907-95aeb0d18011 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:06.726113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-44b6e5ce-5feb-469d-9662-423f993ea145 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:06.726113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:06.726113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:06.726113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:06.726113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:06.726113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:06.726113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:06.726113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:06.726113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:06.726113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:06.726113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:06.726113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:06.726113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:06.726113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:06.726113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:06.727994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:06.727994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:06.727994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:06.727994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:06.727994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:06.727994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:06.727994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:06.727994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:06.727994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:06.727994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:06.727994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:06.727994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:06.727994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:06.727994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:06.727994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:06.727994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:06.727994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:06.729713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:06.729713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:06.729713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:06.729713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:06.729713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:06.729713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:06.729713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:06.729713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:06.729713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 310/623] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:06.733539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6e5ce-5feb-469d-9662-423f993ea145 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:06.970611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db79bf-1d4a-4e57-a907-95aeb0d18011 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:06.989204 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:07.012691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6e5ce-5feb-469d-9662-423f993ea145 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:07.035124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59cee5-5404-4dc2-b5bb-4b2ff4f2e1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:07.037242 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59cee5-5404-4dc2-b5bb-4b2ff4f2e1ee None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:07.038203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59cee5-5404-4dc2-b5bb-4b2ff4f2e1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:07.038973 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59cee5-5404-4dc2-b5bb-4b2ff4f2e1ee None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:07.040485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59cee5-5404-4dc2-b5bb-4b2ff4f2e1ee None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:07.041321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59cee5-5404-4dc2-b5bb-4b2ff4f2e1ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:07.044126 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59cee5-5404-4dc2-b5bb-4b2ff4f2e1ee None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:07.060923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 314/624] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:01:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:07.069744 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59cee5-5404-4dc2-b5bb-4b2ff4f2e1ee None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:07.080716 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88d670-1fff-4034-a3b4-681c57b4eb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:07.081326 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88d670-1fff-4034-a3b4-681c57b4eb09 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:07.081607 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88d670-1fff-4034-a3b4-681c57b4eb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:07.081880 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88d670-1fff-4034-a3b4-681c57b4eb09 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:07.082584 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88d670-1fff-4034-a3b4-681c57b4eb09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:07.082964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88d670-1fff-4034-a3b4-681c57b4eb09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:07.084916 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88d670-1fff-4034-a3b4-681c57b4eb09 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:07.090007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:07.102819 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 311/625] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:07.105183 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88d670-1fff-4034-a3b4-681c57b4eb09 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:07.236206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88d670-1fff-4034-a3b4-681c57b4eb09 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:07.248689 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59cee5-5404-4dc2-b5bb-4b2ff4f2e1ee None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:07.279876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee19808-3ee5-4de0-9ee9-006cdb287568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:07.281303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee19808-3ee5-4de0-9ee9-006cdb287568 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:07.282119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee19808-3ee5-4de0-9ee9-006cdb287568 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:07.282624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee19808-3ee5-4de0-9ee9-006cdb287568 None admin] PATH_INFO: `/v3/projects/0138d62d5879495499a17e20310d4f61/users/ece176d412574790a13d03dcca08f91a/roles/56d621cf28924933b6356795a6f8ba23` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:07.292783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fb2f9-3691-4cee-9243-f9c5b8e7a6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:07.293426 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fb2f9-3691-4cee-9243-f9c5b8e7a6da None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:07.293702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fb2f9-3691-4cee-9243-f9c5b8e7a6da None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:07.293974 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fb2f9-3691-4cee-9243-f9c5b8e7a6da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:07.305309 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee19808-3ee5-4de0-9ee9-006cdb287568 None admin] RBAC: Authorizing `identity:create_grant(project_id=0138d62d5879495499a17e20310d4f61, user_id=ece176d412574790a13d03dcca08f91a, role_id=56d621cf28924933b6356795a6f8ba23)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:07.306803 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fb2f9-3691-4cee-9243-f9c5b8e7a6da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:07.307735 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee19808-3ee5-4de0-9ee9-006cdb287568 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:07.308289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fb2f9-3691-4cee-9243-f9c5b8e7a6da None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:07.334699 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 312/626] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:01:07 2022] PUT /identity/v3/projects/0138d62d5879495499a17e20310d4f61/users/ece176d412574790a13d03dcca08f91a/roles/56d621cf28924933b6356795a6f8ba23 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:01:07.334699 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee19808-3ee5-4de0-9ee9-006cdb287568 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:07.335018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-599fb2f9-3691-4cee-9243-f9c5b8e7a6da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:07.335018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:07.335018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:07.335018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:07.335018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:07.335018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:07.335018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:07.335018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:07.335018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:07.335018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:07.335018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:07.335018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:07.335018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:07.335018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:07.335018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:07.335018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:07.335881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:07.335881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:07.335881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:07.335881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:07.335881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:07.335881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:07.335881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:07.335881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:07.335881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:07.335881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:07.335881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:07.335881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:07.335881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:07.335881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:07.335881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:07.335881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:07.335881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:07.336756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:07.336756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:07.336756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:07.336756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 315/627] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:07.336756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ca73b-adf8-47fd-9bc9-84137c9c2189 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:07.336756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ca73b-adf8-47fd-9bc9-84137c9c2189 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:07.336756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ca73b-adf8-47fd-9bc9-84137c9c2189 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:07.336756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 316/628] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:07.338931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ca73b-adf8-47fd-9bc9-84137c9c2189 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:07.340312 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:07.454089 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:07.513333 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee19808-3ee5-4de0-9ee9-006cdb287568 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:07.521506 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ca73b-adf8-47fd-9bc9-84137c9c2189 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:07.561281 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2fa8d1-1c53-417f-b41f-2cd2a65ad7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:07.562860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fa8d1-1c53-417f-b41f-2cd2a65ad7af None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:07.563446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fa8d1-1c53-417f-b41f-2cd2a65ad7af None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:07.564262 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fa8d1-1c53-417f-b41f-2cd2a65ad7af None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:07.586206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2fa8d1-1c53-417f-b41f-2cd2a65ad7af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:01:07.591112 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea98ba-db5a-424c-8dbf-8e6ad5a06ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:07.592159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea98ba-db5a-424c-8dbf-8e6ad5a06ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:07.592670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea98ba-db5a-424c-8dbf-8e6ad5a06ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:07.593165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea98ba-db5a-424c-8dbf-8e6ad5a06ed9 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:07.594589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea98ba-db5a-424c-8dbf-8e6ad5a06ed9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:07.595195 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea98ba-db5a-424c-8dbf-8e6ad5a06ed9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:07.596114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-9b2fa8d1-1c53-417f-b41f-2cd2a65ad7af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:01:07.596114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:07.596114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:07.596114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:07.596114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:07.596114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:07.596114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:07.596114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:07.596114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:07.596114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:07.596114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:07.596114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:07.596114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:01:07.596114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:01:07.596114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:01:07.596114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:01:07.597387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:01:07.597387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:01:07.597387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:01:07.597387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:01:07.597387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:07.597387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:07.597387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:07.597387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:07.597387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:07.597387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:07.597387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:07.597387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:07.597387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:07.597387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:07.597387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:07.597387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:07.598862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:07.598862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:07.598862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:07.598862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:07.598862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:01:07.598862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:01:07.598862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:01:07.598862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:01:07.598862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:01:07.598862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:01:07.598862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:01:07.598862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:07.600969 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea98ba-db5a-424c-8dbf-8e6ad5a06ed9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:07.605342 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 313/629] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:01:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:07.608262 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1b499-10bf-45c0-baa4-493bf7318c4a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:07.609090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1b499-10bf-45c0-baa4-493bf7318c4a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:07.609837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1b499-10bf-45c0-baa4-493bf7318c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:07.623181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 317/630] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:07.631078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea98ba-db5a-424c-8dbf-8e6ad5a06ed9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:07.653480 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f4d1b499-10bf-45c0-baa4-493bf7318c4a None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:07.655318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f4d1b499-10bf-45c0-baa4-493bf7318c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:07.767005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:07.793234 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 314/631] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:01:07.793234 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1b499-10bf-45c0-baa4-493bf7318c4a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:07.796459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea98ba-db5a-424c-8dbf-8e6ad5a06ed9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:07.846409 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:07.887419 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69bf25-685a-481a-9c18-1346be36a392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:07.888054 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69bf25-685a-481a-9c18-1346be36a392 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:07.888337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69bf25-685a-481a-9c18-1346be36a392 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:07.888613 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69bf25-685a-481a-9c18-1346be36a392 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:07.889299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69bf25-685a-481a-9c18-1346be36a392 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:07.889608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69bf25-685a-481a-9c18-1346be36a392 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:07.891040 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69bf25-685a-481a-9c18-1346be36a392 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:07.921230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 318/632] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:01:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:07.923201 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69bf25-685a-481a-9c18-1346be36a392 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:07.935939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1b499-10bf-45c0-baa4-493bf7318c4a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:08.004737 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82daea7-8a41-43c1-b8b1-9a59cd421c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:08.006753 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82daea7-8a41-43c1-b8b1-9a59cd421c76 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:08.007527 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82daea7-8a41-43c1-b8b1-9a59cd421c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:08.008287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82daea7-8a41-43c1-b8b1-9a59cd421c76 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:08.018626 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82daea7-8a41-43c1-b8b1-9a59cd421c76 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:08.020240 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82daea7-8a41-43c1-b8b1-9a59cd421c76 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:08.035419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-d82daea7-8a41-43c1-b8b1-9a59cd421c76 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 16:01:08.035419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:08.035419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:08.035419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:08.035419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:08.035419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:08.035419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:08.035419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:08.035419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:08.035419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:08.035419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:08.035419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:08.035419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:08.035419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:08.035419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:08.035419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:08.038059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:08.038059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:08.038059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:08.038059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:08.038059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:08.038059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:08.038059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:08.038059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:08.038059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:08.038059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:08.038059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:08.038059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:08.038059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:08.038059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:08.038059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:08.038059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:08.038059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:08.038059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:08.040144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:08.040144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:08.040144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:08.040144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:08.040144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:08.040144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:08.040144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:08.040144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:08.040144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 16:01:08.040144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:08.040144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 315/633] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:01:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:08.046385 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8cbd7-cf96-4e75-a46e-258b4d52c870 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:08.046811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8cbd7-cf96-4e75-a46e-258b4d52c870 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:08.047215 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8cbd7-cf96-4e75-a46e-258b4d52c870 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:08.053963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 316/634] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:08.053963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8cbd7-cf96-4e75-a46e-258b4d52c870 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:08.111840 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69bf25-685a-481a-9c18-1346be36a392 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:08.146249 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:08.160250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2078d3-d78d-4939-ae2d-1b6c0d84c1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:08.162211 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2078d3-d78d-4939-ae2d-1b6c0d84c1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:08.162703 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2078d3-d78d-4939-ae2d-1b6c0d84c1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:08.163176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2078d3-d78d-4939-ae2d-1b6c0d84c1f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:08.176059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2078d3-d78d-4939-ae2d-1b6c0d84c1f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:08.178293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2078d3-d78d-4939-ae2d-1b6c0d84c1f8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:08.193854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-1f2078d3-d78d-4939-ae2d-1b6c0d84c1f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:08.193854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:08.193854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:08.193854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:08.193854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:08.193854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:08.193854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:08.193854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:08.193854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:08.193854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:08.193854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:08.193854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:08.193854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:08.193854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:08.193854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:08.193854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:08.195388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:08.195388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:08.195388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:08.195388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:08.195388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:08.195388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:08.195388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:08.195388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:08.195388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:08.195388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:08.195388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:08.195388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:08.195388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:08.195388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:08.195388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:08.195388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:08.195388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:08.196480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:08.196480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:08.196480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:08.203399 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 319/635] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:08.203399 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489776e1-f5df-4877-bdad-b74d81a57a99 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:08.203399 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489776e1-f5df-4877-bdad-b74d81a57a99 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:08.203399 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489776e1-f5df-4877-bdad-b74d81a57a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:08.220526 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-489776e1-f5df-4877-bdad-b74d81a57a99 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:08.221842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-489776e1-f5df-4877-bdad-b74d81a57a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:08.233187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8cbd7-cf96-4e75-a46e-258b4d52c870 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:08.253511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:08.277068 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2944d-be88-4280-98d2-374500a21d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:08.277877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2944d-be88-4280-98d2-374500a21d3a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:08.278220 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2944d-be88-4280-98d2-374500a21d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:08.278558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2944d-be88-4280-98d2-374500a21d3a None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:08.279427 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce2944d-be88-4280-98d2-374500a21d3a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:08.279787 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce2944d-be88-4280-98d2-374500a21d3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:08.281585 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce2944d-be88-4280-98d2-374500a21d3a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:08.311095 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 317/636] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Sep 6 16:01:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:08.311095 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2944d-be88-4280-98d2-374500a21d3a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:08.325967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 320/637] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:01:08.327574 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489776e1-f5df-4877-bdad-b74d81a57a99 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:08.469720 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2944d-be88-4280-98d2-374500a21d3a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:08.496991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:08.545954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71c90d-4e74-4034-b640-e4ca9f528716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:08.547476 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71c90d-4e74-4034-b640-e4ca9f528716 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:08.548215 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71c90d-4e74-4034-b640-e4ca9f528716 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:08.548957 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71c90d-4e74-4034-b640-e4ca9f528716 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:08.550025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489776e1-f5df-4877-bdad-b74d81a57a99 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:08.570831 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71c90d-4e74-4034-b640-e4ca9f528716 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:01:08.581751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-4a71c90d-4e74-4034-b640-e4ca9f528716 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:01:08.581751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:08.581751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:08.581751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:08.581751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:08.581751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:08.581751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:08.581751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:08.581751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:08.581751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:08.581751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:08.581751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:08.581751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:01:08.581751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:01:08.581751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:01:08.581751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:01:08.584060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:01:08.584060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:01:08.584060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:01:08.584060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:01:08.584060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:08.584060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:08.584060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:08.584060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:08.584060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:08.584060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:08.584060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:08.584060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:08.584060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:08.584060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:08.584060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:08.584060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:08.585731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:08.585731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:08.585731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:08.585731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:08.585731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:01:08.585731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:01:08.585731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:01:08.585731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:01:08.585731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:01:08.585731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:01:08.585731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:01:08.585731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:08.590509 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 318/638] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:01:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:08.592393 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71c90d-4e74-4034-b640-e4ca9f528716 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:08.631745 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13fee1-0c6d-4329-b2a8-e8773cd1ad22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:08.632530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13fee1-0c6d-4329-b2a8-e8773cd1ad22 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:08.632905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13fee1-0c6d-4329-b2a8-e8773cd1ad22 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:08.633362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13fee1-0c6d-4329-b2a8-e8773cd1ad22 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:08.634039 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13fee1-0c6d-4329-b2a8-e8773cd1ad22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:08.634440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13fee1-0c6d-4329-b2a8-e8773cd1ad22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:08.635984 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13fee1-0c6d-4329-b2a8-e8773cd1ad22 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:08.649948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 321/639] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:08.650728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13fee1-0c6d-4329-b2a8-e8773cd1ad22 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:08.687933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:08.742726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71c90d-4e74-4034-b640-e4ca9f528716 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:08.798147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60b723-7fb7-424a-a358-26edc4a4b12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:08.799126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60b723-7fb7-424a-a358-26edc4a4b12a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:08.799618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60b723-7fb7-424a-a358-26edc4a4b12a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:08.800095 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60b723-7fb7-424a-a358-26edc4a4b12a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:08.810496 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13fee1-0c6d-4329-b2a8-e8773cd1ad22 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:08.871534 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:08.876725 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123472d-3b16-4973-81c4-d4eb72eab3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:08.886830 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123472d-3b16-4973-81c4-d4eb72eab3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:08.886830 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123472d-3b16-4973-81c4-d4eb72eab3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:08.886830 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123472d-3b16-4973-81c4-d4eb72eab3e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:08.886830 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123472d-3b16-4973-81c4-d4eb72eab3e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:08.886830 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123472d-3b16-4973-81c4-d4eb72eab3e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:08.886830 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123472d-3b16-4973-81c4-d4eb72eab3e7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:08.902685 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 322/640] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:01:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:08.904077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123472d-3b16-4973-81c4-d4eb72eab3e7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:08.930388 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60b723-7fb7-424a-a358-26edc4a4b12a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:08.931935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60b723-7fb7-424a-a358-26edc4a4b12a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:09.034641 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:09.070321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123472d-3b16-4973-81c4-d4eb72eab3e7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:09.088405 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 319/641] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:01:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:01:09.089298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60b723-7fb7-424a-a358-26edc4a4b12a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:09.140741 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35420388-7901-48d8-af24-6a23216e40fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:09.141813 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35420388-7901-48d8-af24-6a23216e40fb None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:09.142305 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35420388-7901-48d8-af24-6a23216e40fb None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:09.142844 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35420388-7901-48d8-af24-6a23216e40fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:09.161938 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35420388-7901-48d8-af24-6a23216e40fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:01:09.172991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-35420388-7901-48d8-af24-6a23216e40fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:01:09.172991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:09.172991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:09.172991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:09.172991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:09.172991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:09.172991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:09.172991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:09.172991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:09.172991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:09.172991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:09.172991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:09.172991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:01:09.172991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:01:09.172991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:01:09.172991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:01:09.175043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:01:09.175043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:01:09.175043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:01:09.175043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:01:09.175043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:09.175043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:09.175043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:09.175043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:09.175043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:09.175043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:09.175043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:09.175043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:09.175043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:09.175043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:09.175043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:09.175043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:09.178459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:09.178459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:09.178459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:09.178459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:09.178459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:01:09.178459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:01:09.178459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:01:09.178459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:01:09.178459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:01:09.178459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:01:09.178459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:01:09.178459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:09.178459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 323/642] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:01:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:09.178459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35420388-7901-48d8-af24-6a23216e40fb None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:09.187924 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:09.233398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60b723-7fb7-424a-a358-26edc4a4b12a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:09.276554 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64da6d8-5149-44b8-b265-db44011beb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:09.277200 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64da6d8-5149-44b8-b265-db44011beb99 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:09.278131 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64da6d8-5149-44b8-b265-db44011beb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:09.278131 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64da6d8-5149-44b8-b265-db44011beb99 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:09.287053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64da6d8-5149-44b8-b265-db44011beb99 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:09.288546 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64da6d8-5149-44b8-b265-db44011beb99 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:09.301903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-f64da6d8-5149-44b8-b265-db44011beb99 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:01:09.301903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:09.301903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:09.301903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:09.301903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:09.301903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:09.301903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:09.301903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:09.301903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:09.301903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:09.301903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:09.301903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:09.301903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:09.301903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:09.301903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:09.301903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:09.305706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:09.305706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:09.305706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:09.305706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:09.305706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:09.305706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:09.305706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:09.305706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:09.305706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:09.305706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:09.305706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:09.305706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:09.305706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:09.305706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:09.305706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:09.305706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:09.305706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:09.305706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:09.308305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:09.308305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:09.308305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:09.308305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:09.308305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:09.308305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:09.308305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:09.308305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:09.308305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:01:09.308305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:09.308305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 320/643] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Sep 6 16:01:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:09.308305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64da6d8-5149-44b8-b265-db44011beb99 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:09.379865 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35420388-7901-48d8-af24-6a23216e40fb None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:09.439588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd42e54-3670-49ad-9291-894745de1055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:09.440944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd42e54-3670-49ad-9291-894745de1055 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:09.441391 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd42e54-3670-49ad-9291-894745de1055 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:09.441809 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd42e54-3670-49ad-9291-894745de1055 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:09.460578 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd42e54-3670-49ad-9291-894745de1055 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:09.463211 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd42e54-3670-49ad-9291-894745de1055 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:09.468151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:09.487349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64da6d8-5149-44b8-b265-db44011beb99 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:09.492125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-4fd42e54-3670-49ad-9291-894745de1055 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:09.492125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:09.492125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:09.492125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:09.492125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:09.492125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:09.492125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:09.492125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:09.492125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:09.492125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:09.492125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:09.492125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:09.492125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:09.492125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:09.492125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:09.492125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:09.493440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:09.493440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:09.493440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:09.493440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:09.493440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:09.493440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:09.493440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:09.493440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:09.493440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:09.493440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:09.493440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:09.493440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:09.493440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:09.493440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:09.493440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:09.493440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:09.493440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:09.495779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:09.495779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:09.495779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:09.495779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 324/644] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:09.495779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd42e54-3670-49ad-9291-894745de1055 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:09.533054 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:09.540534 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9ef9b-b018-45cb-9c81-8a8534907533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:09.541168 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9ef9b-b018-45cb-9c81-8a8534907533 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:09.541448 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9ef9b-b018-45cb-9c81-8a8534907533 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:09.541717 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9ef9b-b018-45cb-9c81-8a8534907533 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:09.554477 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9ef9b-b018-45cb-9c81-8a8534907533 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:09.555973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9ef9b-b018-45cb-9c81-8a8534907533 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:09.572528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-00a9ef9b-b018-45cb-9c81-8a8534907533 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:09.572528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:09.572528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:09.572528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:09.572528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:09.572528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:09.572528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:09.572528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:09.572528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:09.572528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:09.572528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:09.572528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:09.572528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:09.572528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:09.572528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:09.572528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:09.573678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:09.573678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:09.573678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:09.573678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:09.573678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:09.573678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:09.573678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:09.573678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:09.573678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:09.573678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:09.573678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:09.573678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:09.573678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:09.573678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:09.573678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:09.573678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:09.573678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:09.574546 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:09.574546 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:09.574546 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:09.574546 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 321/645] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:09.574546 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9ef9b-b018-45cb-9c81-8a8534907533 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:09.645893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd42e54-3670-49ad-9291-894745de1055 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:09.700352 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bfbe7-f992-48d6-8829-37705356f778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:09.701246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bfbe7-f992-48d6-8829-37705356f778 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:09.701746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bfbe7-f992-48d6-8829-37705356f778 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:09.702238 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bfbe7-f992-48d6-8829-37705356f778 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:09.703469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9ef9b-b018-45cb-9c81-8a8534907533 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:09.703583 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bfbe7-f992-48d6-8829-37705356f778 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:09.704039 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bfbe7-f992-48d6-8829-37705356f778 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:09.706417 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bfbe7-f992-48d6-8829-37705356f778 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:09.726308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 325/646] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:01:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:09.726308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ea62f-6d01-43ac-9a50-9485c20bf273 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:09.726308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ea62f-6d01-43ac-9a50-9485c20bf273 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:09.726308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ea62f-6d01-43ac-9a50-9485c20bf273 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:09.727818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 326/647] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:09.729144 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ea62f-6d01-43ac-9a50-9485c20bf273 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:09.755890 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd0824-1e30-47b3-b43c-8f5b3e87066a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:09.756975 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd0824-1e30-47b3-b43c-8f5b3e87066a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:09.757477 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd0824-1e30-47b3-b43c-8f5b3e87066a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:09.757963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd0824-1e30-47b3-b43c-8f5b3e87066a None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:09.759498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd0824-1e30-47b3-b43c-8f5b3e87066a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:09.760062 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd0824-1e30-47b3-b43c-8f5b3e87066a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:09.762791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd0824-1e30-47b3-b43c-8f5b3e87066a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:09.795242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 322/648] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Sep 6 16:01:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:09.796814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd0824-1e30-47b3-b43c-8f5b3e87066a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:09.807295 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:09.846608 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:09.866750 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ea62f-6d01-43ac-9a50-9485c20bf273 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:09.908025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634c2de-4f7c-42ed-80f2-494eab4e6a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:09.908578 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634c2de-4f7c-42ed-80f2-494eab4e6a9f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:09.908860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634c2de-4f7c-42ed-80f2-494eab4e6a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:09.909138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634c2de-4f7c-42ed-80f2-494eab4e6a9f None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:09.909792 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2634c2de-4f7c-42ed-80f2-494eab4e6a9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:09.910095 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2634c2de-4f7c-42ed-80f2-494eab4e6a9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:09.912135 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2634c2de-4f7c-42ed-80f2-494eab4e6a9f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:09.913060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd0824-1e30-47b3-b43c-8f5b3e87066a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:09.927803 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 327/649] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:09.929094 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634c2de-4f7c-42ed-80f2-494eab4e6a9f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:09.963514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49519ac-8c99-49b6-873b-6207ba68463c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:09.963514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49519ac-8c99-49b6-873b-6207ba68463c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:09.963514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49519ac-8c99-49b6-873b-6207ba68463c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:09.963514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49519ac-8c99-49b6-873b-6207ba68463c None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:09.964103 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49519ac-8c99-49b6-873b-6207ba68463c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:09.964103 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49519ac-8c99-49b6-873b-6207ba68463c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:09.965243 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49519ac-8c99-49b6-873b-6207ba68463c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:09.979441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 323/650] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:09.980820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdde56c-96fb-4c4f-bbd9-df7a6667ad2e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:09.981113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdde56c-96fb-4c4f-bbd9-df7a6667ad2e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:09.981386 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdde56c-96fb-4c4f-bbd9-df7a6667ad2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:10.010550 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-cbdde56c-96fb-4c4f-bbd9-df7a6667ad2e None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:10.011845 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cbdde56c-96fb-4c4f-bbd9-df7a6667ad2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:10.079166 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634c2de-4f7c-42ed-80f2-494eab4e6a9f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:10.092005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 324/651] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:01:10.092886 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdde56c-96fb-4c4f-bbd9-df7a6667ad2e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:10.117494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:10.129115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:10.155501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153de713-a165-44a7-8636-958b84f55e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:10.156547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153de713-a165-44a7-8636-958b84f55e4f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:10.156978 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153de713-a165-44a7-8636-958b84f55e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:10.157436 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153de713-a165-44a7-8636-958b84f55e4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:10.178174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153de713-a165-44a7-8636-958b84f55e4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:10.180967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153de713-a165-44a7-8636-958b84f55e4f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:10.210473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-153de713-a165-44a7-8636-958b84f55e4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:10.210473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:10.210473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:10.210473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:10.210473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:10.210473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:10.210473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:10.210473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:10.210473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:10.210473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:10.210473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:10.210473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:10.210473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:10.210473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:10.210473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:10.210473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:10.212258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:10.212258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:10.212258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:10.212258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:10.212258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:10.212258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:10.212258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:10.212258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:10.212258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:10.212258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:10.212258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:10.212258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:10.212258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:10.212258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:10.212258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:10.212258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:10.212258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:10.215203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:10.215203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:10.215203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:10.215203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 328/652] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:10.215203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153de713-a165-44a7-8636-958b84f55e4f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:10.227527 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdde56c-96fb-4c4f-bbd9-df7a6667ad2e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:10.284388 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50f03b-d3fe-44ee-9e34-925dfffa9a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:10.285370 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50f03b-d3fe-44ee-9e34-925dfffa9a78 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:10.285370 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50f03b-d3fe-44ee-9e34-925dfffa9a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:10.285626 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50f03b-d3fe-44ee-9e34-925dfffa9a78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:10.383839 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50f03b-d3fe-44ee-9e34-925dfffa9a78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:10.385924 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50f03b-d3fe-44ee-9e34-925dfffa9a78 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:10.386322 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153de713-a165-44a7-8636-958b84f55e4f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:10.429717 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56243854-aabd-42a1-a389-647b7b4960a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:10.430627 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56243854-aabd-42a1-a389-647b7b4960a6 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:10.431135 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56243854-aabd-42a1-a389-647b7b4960a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:10.431413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56243854-aabd-42a1-a389-647b7b4960a6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:10.433166 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:10.445192 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56243854-aabd-42a1-a389-647b7b4960a6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:10.448058 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56243854-aabd-42a1-a389-647b7b4960a6 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:10.461778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-56243854-aabd-42a1-a389-647b7b4960a6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 16:01:10.461778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:10.461778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:10.461778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:10.461778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:10.461778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:10.461778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:10.461778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:10.461778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:10.461778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:10.461778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:10.461778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:10.461778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:10.461778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:10.461778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:10.461778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:10.463237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:10.463237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:10.463237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:10.463237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:10.463237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:10.463237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:10.463237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:10.463237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:10.463237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:10.463237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:10.463237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:10.463237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:10.463237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:10.463237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:10.463237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:10.463237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:10.463237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:10.463237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:10.465108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:10.465108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:10.465108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:10.465108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:10.465108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:10.465108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:10.465108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:10.465108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:10.465108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 16:01:10.465108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:10.465108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 329/653] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:01:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:10.465108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56243854-aabd-42a1-a389-647b7b4960a6 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:10.499817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:10.535511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 325/654] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:01:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:01:10.537089 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50f03b-d3fe-44ee-9e34-925dfffa9a78 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:10.611115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56243854-aabd-42a1-a389-647b7b4960a6 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:10.654880 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50f03b-d3fe-44ee-9e34-925dfffa9a78 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:10.658444 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136170e3-cb75-449c-869e-4396c801fa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:10.659134 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136170e3-cb75-449c-869e-4396c801fa8b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:10.659415 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136170e3-cb75-449c-869e-4396c801fa8b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:10.659681 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136170e3-cb75-449c-869e-4396c801fa8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:10.667413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136170e3-cb75-449c-869e-4396c801fa8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:10.668920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136170e3-cb75-449c-869e-4396c801fa8b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:10.682213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-136170e3-cb75-449c-869e-4396c801fa8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:10.682213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:10.682213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:10.682213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:10.682213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:10.682213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:10.682213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:10.682213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:10.682213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:10.682213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:10.682213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:10.682213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:10.682213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:10.682213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:10.682213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:10.683948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:10.683948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:10.683948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:10.683948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:10.683948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:10.683948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:10.683948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:10.683948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:10.683948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:10.683948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:10.683948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:10.683948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:10.683948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:10.683948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:10.683948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:10.683948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:10.683948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:10.685696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:10.685696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:10.685696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:10.685696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:10.685696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:10.685696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:10.685696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:10.685696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:10.685696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 330/655] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:10.685696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136170e3-cb75-449c-869e-4396c801fa8b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:10.691154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:10.698017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3f3aa-e01a-4775-be00-c71424a09771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:10.699024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3f3aa-e01a-4775-be00-c71424a09771 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:10.699315 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3f3aa-e01a-4775-be00-c71424a09771 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:10.699583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3f3aa-e01a-4775-be00-c71424a09771 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:10.700556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3f3aa-e01a-4775-be00-c71424a09771 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:10.700860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3f3aa-e01a-4775-be00-c71424a09771 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:10.702229 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3f3aa-e01a-4775-be00-c71424a09771 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:10.714695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 326/656] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:10.715628 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3f3aa-e01a-4775-be00-c71424a09771 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:10.787931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:10.799044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136170e3-cb75-449c-869e-4396c801fa8b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:10.836373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3f3aa-e01a-4775-be00-c71424a09771 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:10.841053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521ae4e-2e0a-440c-bcd5-a47db771c399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:10.841695 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521ae4e-2e0a-440c-bcd5-a47db771c399 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:10.841966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521ae4e-2e0a-440c-bcd5-a47db771c399 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:10.842240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521ae4e-2e0a-440c-bcd5-a47db771c399 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:10.854030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a521ae4e-2e0a-440c-bcd5-a47db771c399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:01:10.860158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-a521ae4e-2e0a-440c-bcd5-a47db771c399 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:01:10.860158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:10.860158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:10.860158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:10.860158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:10.860158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:10.860158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:10.860158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:10.860158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:10.860158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:10.860158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:10.860158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:10.860158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:01:10.860158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:01:10.860158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:01:10.860158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:01:10.861986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:01:10.861986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:01:10.861986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:01:10.861986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:01:10.861986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:10.861986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:10.861986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:10.861986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:10.861986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:10.861986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:10.861986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:10.861986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:10.861986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:10.861986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:10.861986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:10.861986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:10.863683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:10.863683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:10.863683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:10.863683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:10.863683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:01:10.863683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:01:10.863683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:01:10.863683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:01:10.863683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:01:10.863683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:01:10.863683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:01:10.863683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:10.863683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 331/657] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:01:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:10.863683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521ae4e-2e0a-440c-bcd5-a47db771c399 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:10.879015 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd2b0d-f390-408b-8c9e-89e6937880a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:10.879628 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd2b0d-f390-408b-8c9e-89e6937880a1 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:10.879909 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd2b0d-f390-408b-8c9e-89e6937880a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:10.880785 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd2b0d-f390-408b-8c9e-89e6937880a1 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:10.881977 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd2b0d-f390-408b-8c9e-89e6937880a1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:10.882292 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd2b0d-f390-408b-8c9e-89e6937880a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:10.883786 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd2b0d-f390-408b-8c9e-89e6937880a1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:10.902893 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 327/658] 10.222.0.26 () {56 vars in 1139 bytes} [Tue Sep 6 16:01:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:10.903752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd2b0d-f390-408b-8c9e-89e6937880a1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:10.961680 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:11.016419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd2b0d-f390-408b-8c9e-89e6937880a1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:11.044174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521ae4e-2e0a-440c-bcd5-a47db771c399 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:11.056782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c89d4a-50d0-4180-a8fb-3a2d097555c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:11.057396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c89d4a-50d0-4180-a8fb-3a2d097555c6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:11.057676 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c89d4a-50d0-4180-a8fb-3a2d097555c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:11.057958 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c89d4a-50d0-4180-a8fb-3a2d097555c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:11.058690 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c89d4a-50d0-4180-a8fb-3a2d097555c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:11.059004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c89d4a-50d0-4180-a8fb-3a2d097555c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:11.060450 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c89d4a-50d0-4180-a8fb-3a2d097555c6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:11.073735 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 328/659] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:11.074601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c89d4a-50d0-4180-a8fb-3a2d097555c6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:11.102458 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3523eb-c01e-4d19-b795-9a1d2a8ea6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:11.103241 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3523eb-c01e-4d19-b795-9a1d2a8ea6ce None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:11.103660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3523eb-c01e-4d19-b795-9a1d2a8ea6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:11.103996 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3523eb-c01e-4d19-b795-9a1d2a8ea6ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:11.119522 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3523eb-c01e-4d19-b795-9a1d2a8ea6ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:11.121244 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3523eb-c01e-4d19-b795-9a1d2a8ea6ce None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:11.138530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-1f3523eb-c01e-4d19-b795-9a1d2a8ea6ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:11.138530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:11.138530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:11.138530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:11.138530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:11.138530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:11.138530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:11.138530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:11.138530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:11.138530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:11.138530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:11.138530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:11.138530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:11.138530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:11.138530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:11.138530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:11.139670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:11.139670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:11.139670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:11.139670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:11.139670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:11.139670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:11.139670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:11.139670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:11.139670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:11.139670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:11.139670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:11.139670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:11.139670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:11.139670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:11.139670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:11.139670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:11.139670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:11.140727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:11.140727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:11.140727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:11.142625 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 332/660] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:11.143251 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3523eb-c01e-4d19-b795-9a1d2a8ea6ce None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:11.146931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:11.211850 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c89d4a-50d0-4180-a8fb-3a2d097555c6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:11.223089 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:11.263051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77b8e0-3197-4f89-8058-964945614591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:11.263693 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77b8e0-3197-4f89-8058-964945614591 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:11.263986 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77b8e0-3197-4f89-8058-964945614591 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:11.264257 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77b8e0-3197-4f89-8058-964945614591 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:11.266643 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3523eb-c01e-4d19-b795-9a1d2a8ea6ce None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:11.274142 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77b8e0-3197-4f89-8058-964945614591 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:11.275679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77b8e0-3197-4f89-8058-964945614591 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:11.288770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-1c77b8e0-3197-4f89-8058-964945614591 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 16:01:11.288770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:11.288770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:11.288770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:11.288770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:11.288770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:11.288770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:11.288770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:11.288770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:11.288770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:11.288770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:11.288770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:11.288770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:11.288770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:11.288770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:11.288770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:11.291401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:11.291401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:11.291401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:11.291401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:11.291401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:11.291401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:11.291401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:11.291401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:11.291401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:11.291401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:11.291401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:11.291401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:11.291401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:11.291401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:11.291401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:11.291401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:11.291401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:11.291401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:11.293296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:11.293296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:11.293296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:11.293296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:11.293296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:11.293296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:11.293296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:11.293296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:11.293296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 16:01:11.293296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:11.293296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 329/661] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:01:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:11.293296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77b8e0-3197-4f89-8058-964945614591 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:11.321523 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b9734-0ede-479c-a759-994dee4fc662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:11.322289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b9734-0ede-479c-a759-994dee4fc662 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:11.322705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b9734-0ede-479c-a759-994dee4fc662 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:11.323072 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b9734-0ede-479c-a759-994dee4fc662 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:11.323920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b9734-0ede-479c-a759-994dee4fc662 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:11.324324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b9734-0ede-479c-a759-994dee4fc662 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:11.325827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b9734-0ede-479c-a759-994dee4fc662 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:11.335515 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 333/662] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:01:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:11.338907 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b9734-0ede-479c-a759-994dee4fc662 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:11.415268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:11.429520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77b8e0-3197-4f89-8058-964945614591 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:11.465444 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b9734-0ede-479c-a759-994dee4fc662 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:11.481265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06af89-1114-4c39-a906-ec95023d4dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:11.482249 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06af89-1114-4c39-a906-ec95023d4dcf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:11.482788 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06af89-1114-4c39-a906-ec95023d4dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:11.483269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06af89-1114-4c39-a906-ec95023d4dcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:11.504851 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10960c3f-7732-4000-a47b-0e82b13d1b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:11.506492 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10960c3f-7732-4000-a47b-0e82b13d1b1d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:11.506743 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10960c3f-7732-4000-a47b-0e82b13d1b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:11.506918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10960c3f-7732-4000-a47b-0e82b13d1b1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:11.507660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10960c3f-7732-4000-a47b-0e82b13d1b1d None admin] RBAC: Adding query filter params (role.id=56d621cf28924933b6356795a6f8ba23, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=e50ef7db27744c7ebd2ec95cecc9d1a8) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:11.507968 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10960c3f-7732-4000-a47b-0e82b13d1b1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:11.509417 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10960c3f-7732-4000-a47b-0e82b13d1b1d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:11.515838 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 334/663] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:01:11 2022] GET /identity/v3/role_assignments?user.id=e50ef7db27744c7ebd2ec95cecc9d1a8&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=56d621cf28924933b6356795a6f8ba23 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:11.516675 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10960c3f-7732-4000-a47b-0e82b13d1b1d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:11.562129 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:11.632122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06af89-1114-4c39-a906-ec95023d4dcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:11.634639 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06af89-1114-4c39-a906-ec95023d4dcf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:11.698382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10960c3f-7732-4000-a47b-0e82b13d1b1d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:11.742414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:11.755683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe63839-b692-4fc9-960e-3e446943369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:11.756326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe63839-b692-4fc9-960e-3e446943369e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:11.756797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe63839-b692-4fc9-960e-3e446943369e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:11.757061 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe63839-b692-4fc9-960e-3e446943369e None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:11.758296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe63839-b692-4fc9-960e-3e446943369e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:11.758825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe63839-b692-4fc9-960e-3e446943369e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:11.760490 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe63839-b692-4fc9-960e-3e446943369e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:11.775419 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 335/664] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:11.776378 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe63839-b692-4fc9-960e-3e446943369e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:11.791354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 330/665] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:01:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:01:11.794724 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06af89-1114-4c39-a906-ec95023d4dcf None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:11.915194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:11.960873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06af89-1114-4c39-a906-ec95023d4dcf None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:11.968789 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe63839-b692-4fc9-960e-3e446943369e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:12.021603 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08cf52b-c404-41c4-90f4-2edc95417625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:12.022579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cf52b-c404-41c4-90f4-2edc95417625 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:12.023109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cf52b-c404-41c4-90f4-2edc95417625 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:12.023601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cf52b-c404-41c4-90f4-2edc95417625 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:12.038629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be0e43-cb1a-4fdc-9d51-3d81eb278dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:12.039231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be0e43-cb1a-4fdc-9d51-3d81eb278dee None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:12.039681 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be0e43-cb1a-4fdc-9d51-3d81eb278dee None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:12.040124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be0e43-cb1a-4fdc-9d51-3d81eb278dee None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:12.041370 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be0e43-cb1a-4fdc-9d51-3d81eb278dee None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:12.041859 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be0e43-cb1a-4fdc-9d51-3d81eb278dee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:12.042049 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08cf52b-c404-41c4-90f4-2edc95417625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:12.044350 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be0e43-cb1a-4fdc-9d51-3d81eb278dee None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:12.044464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08cf52b-c404-41c4-90f4-2edc95417625 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:12.071516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-e08cf52b-c404-41c4-90f4-2edc95417625 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:12.071516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:12.071516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:12.071516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:12.071516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:12.071516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:12.071516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:12.071516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:12.071516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:12.071516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:12.071516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:12.071516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:12.071516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:12.071516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:12.071516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:12.071516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:12.072514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:12.072514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:12.072514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:12.072514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:12.072514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:12.072514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:12.072514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:12.072514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:12.072514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:12.072514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:12.072514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:12.072514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:12.072514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:12.072514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:12.072514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:12.072514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:12.072514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:12.073379 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:12.073379 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:12.073379 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:12.073379 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 331/666] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:12.074943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08cf52b-c404-41c4-90f4-2edc95417625 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:12.084506 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 336/667] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Sep 6 16:01:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:12.085838 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be0e43-cb1a-4fdc-9d51-3d81eb278dee None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:12.163210 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:12.236726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08cf52b-c404-41c4-90f4-2edc95417625 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:12.252542 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be0e43-cb1a-4fdc-9d51-3d81eb278dee None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:12.266131 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:12.288632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b26585-b6a6-4f42-bb71-56a7b7a89442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:12.289160 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b26585-b6a6-4f42-bb71-56a7b7a89442 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:12.289728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b26585-b6a6-4f42-bb71-56a7b7a89442 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:12.289728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b26585-b6a6-4f42-bb71-56a7b7a89442 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:12.298037 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b26585-b6a6-4f42-bb71-56a7b7a89442 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:12.299553 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b26585-b6a6-4f42-bb71-56a7b7a89442 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:12.302771 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a864c-01fd-47fa-9356-e00100ef6816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:12.303524 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a864c-01fd-47fa-9356-e00100ef6816 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:12.303665 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a864c-01fd-47fa-9356-e00100ef6816 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:12.304510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a864c-01fd-47fa-9356-e00100ef6816 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:12.314179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-60b26585-b6a6-4f42-bb71-56a7b7a89442 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:12.314179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:12.314179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:12.314179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:12.314179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:12.314179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:12.314179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:12.314179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:12.314179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:12.314179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:12.314179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:12.314179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:12.314179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:12.314179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:12.314179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:12.315280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:12.315280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:12.315280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:12.315280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:12.315280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:12.315280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:12.315280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:12.315280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:12.315280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:12.315280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:12.315280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:12.315280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:12.315280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:12.315280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:12.315280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:12.315280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:12.315280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:12.322113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:12.322113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:12.322113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:12.322113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:12.322113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:12.322113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:12.322113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:12.322113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:12.322113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 332/668] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:12.322113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b26585-b6a6-4f42-bb71-56a7b7a89442 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:12.327277 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a864c-01fd-47fa-9356-e00100ef6816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:12.328795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a864c-01fd-47fa-9356-e00100ef6816 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:12.346365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-b16a864c-01fd-47fa-9356-e00100ef6816 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:12.346365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:12.346365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:12.346365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:12.346365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:12.346365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:12.346365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:12.346365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:12.346365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:12.346365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:12.346365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:12.346365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:12.346365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:12.346365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:12.346365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:12.346365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:12.347421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:12.347421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:12.347421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:12.347421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:12.347421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:12.347421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:12.347421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:12.347421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:12.347421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:12.347421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:12.347421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:12.347421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:12.347421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:12.347421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:12.347421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:12.347421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:12.347421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:12.348292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:12.348292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:12.348292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:12.348292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 337/669] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:12.348514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a864c-01fd-47fa-9356-e00100ef6816 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:12.467655 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b26585-b6a6-4f42-bb71-56a7b7a89442 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:12.476240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:12.481216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a864c-01fd-47fa-9356-e00100ef6816 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:12.512596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c26d46-1d85-4981-b752-43c1957af76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:12.513461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c26d46-1d85-4981-b752-43c1957af76f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:12.513903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c26d46-1d85-4981-b752-43c1957af76f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:12.514317 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c26d46-1d85-4981-b752-43c1957af76f None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:12.515168 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c26d46-1d85-4981-b752-43c1957af76f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:12.515629 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c26d46-1d85-4981-b752-43c1957af76f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:12.517303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c26d46-1d85-4981-b752-43c1957af76f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:12.530047 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd29c5-7a66-4ef3-86d0-1b8a1b369824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:12.530691 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd29c5-7a66-4ef3-86d0-1b8a1b369824 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:12.530984 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd29c5-7a66-4ef3-86d0-1b8a1b369824 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:12.531262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd29c5-7a66-4ef3-86d0-1b8a1b369824 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:12.531666 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 333/670] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:12.532666 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c26d46-1d85-4981-b752-43c1957af76f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:12.593490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:12.660493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd29c5-7a66-4ef3-86d0-1b8a1b369824 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:12.663396 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd29c5-7a66-4ef3-86d0-1b8a1b369824 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:12.672177 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c26d46-1d85-4981-b752-43c1957af76f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:12.721581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2656b6e-2aec-4dfb-9a55-5f707e7e552f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:12.722201 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2656b6e-2aec-4dfb-9a55-5f707e7e552f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:12.722479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2656b6e-2aec-4dfb-9a55-5f707e7e552f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:12.722777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2656b6e-2aec-4dfb-9a55-5f707e7e552f None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:12.723484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2656b6e-2aec-4dfb-9a55-5f707e7e552f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:12.724390 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2656b6e-2aec-4dfb-9a55-5f707e7e552f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:12.726096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2656b6e-2aec-4dfb-9a55-5f707e7e552f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:12.739298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 334/671] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:12.741830 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2656b6e-2aec-4dfb-9a55-5f707e7e552f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:12.801905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:12.813241 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 338/672] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:01:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:01:12.813802 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd29c5-7a66-4ef3-86d0-1b8a1b369824 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:12.901624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:12.906564 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2656b6e-2aec-4dfb-9a55-5f707e7e552f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:12.960766 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75991cc7-17ac-456f-bd1e-3df9d56e1630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:12.962165 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75991cc7-17ac-456f-bd1e-3df9d56e1630 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:12.962908 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75991cc7-17ac-456f-bd1e-3df9d56e1630 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:12.963394 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75991cc7-17ac-456f-bd1e-3df9d56e1630 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:12.964561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75991cc7-17ac-456f-bd1e-3df9d56e1630 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:12.965134 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75991cc7-17ac-456f-bd1e-3df9d56e1630 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:12.967761 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75991cc7-17ac-456f-bd1e-3df9d56e1630 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:12.972600 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd29c5-7a66-4ef3-86d0-1b8a1b369824 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:12.993947 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 335/673] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:12.995858 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75991cc7-17ac-456f-bd1e-3df9d56e1630 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:13.033442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34a2a6-57ee-4a08-9fc8-8485e82e899c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:13.034864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34a2a6-57ee-4a08-9fc8-8485e82e899c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:13.035314 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34a2a6-57ee-4a08-9fc8-8485e82e899c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:13.035753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34a2a6-57ee-4a08-9fc8-8485e82e899c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:13.049111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34a2a6-57ee-4a08-9fc8-8485e82e899c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:13.051615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34a2a6-57ee-4a08-9fc8-8485e82e899c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:13.074389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-dd34a2a6-57ee-4a08-9fc8-8485e82e899c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:01:13.074389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:13.074389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:13.074389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:13.074389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:13.074389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:13.074389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:13.074389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:13.074389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:13.074389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:13.074389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:13.074389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:13.074389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:13.074389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:13.074389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:13.074389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:13.076385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:13.076385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:13.076385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:13.076385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:13.076385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:13.076385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:13.076385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:13.076385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:13.076385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:13.076385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:13.076385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:13.076385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:13.076385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:13.076385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:13.076385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:13.076385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:13.076385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:13.076385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:13.079259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:13.079259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:13.079259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:13.079259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:13.079259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:13.079259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:13.079259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:13.079259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:13.079259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:01:13.079259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:13.079259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 339/674] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:01:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:13.079259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34a2a6-57ee-4a08-9fc8-8485e82e899c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:13.175086 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:13.189006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75991cc7-17ac-456f-bd1e-3df9d56e1630 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:13.246541 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c475e8-84c7-42ac-94cf-15905a9142f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:13.247327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c475e8-84c7-42ac-94cf-15905a9142f7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:13.247704 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c475e8-84c7-42ac-94cf-15905a9142f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:13.248082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c475e8-84c7-42ac-94cf-15905a9142f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:13.248982 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c475e8-84c7-42ac-94cf-15905a9142f7 None admin] RBAC: Adding query filter params (role.id=56d621cf28924933b6356795a6f8ba23, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=6afd4f6ffdae4b16aa5ba757e68c18c4) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:13.249388 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c475e8-84c7-42ac-94cf-15905a9142f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:13.250964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c475e8-84c7-42ac-94cf-15905a9142f7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:13.253559 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34a2a6-57ee-4a08-9fc8-8485e82e899c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:13.267264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 336/675] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:01:12 2022] GET /identity/v3/role_assignments?user.id=6afd4f6ffdae4b16aa5ba757e68c18c4&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=56d621cf28924933b6356795a6f8ba23 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:13.270474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c475e8-84c7-42ac-94cf-15905a9142f7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:13.329406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af094635-8893-4178-9a16-d641b4cd4294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:13.330573 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af094635-8893-4178-9a16-d641b4cd4294 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:13.331218 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af094635-8893-4178-9a16-d641b4cd4294 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:13.331705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af094635-8893-4178-9a16-d641b4cd4294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:13.337037 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:13.356016 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af094635-8893-4178-9a16-d641b4cd4294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:13.359028 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af094635-8893-4178-9a16-d641b4cd4294 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:13.392087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-af094635-8893-4178-9a16-d641b4cd4294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:13.392087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:13.392087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:13.392087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:13.392087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:13.392087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:13.392087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:13.392087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:13.392087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:13.392087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:13.392087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:13.392087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:13.392087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:13.392087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:13.392087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:13.392087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:13.393932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:13.393932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:13.393932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:13.393932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:13.393932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:13.393932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:13.393932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:13.393932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:13.393932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:13.393932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:13.393932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:13.393932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:13.393932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:13.393932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:13.393932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:13.393932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:13.393932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:13.395982 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:13.395982 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:13.395982 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:13.398757 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 340/676] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:13.401754 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af094635-8893-4178-9a16-d641b4cd4294 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:13.495891 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c475e8-84c7-42ac-94cf-15905a9142f7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:13.574944 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad20a0d-8c67-4df2-8536-87579e706fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:13.576805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad20a0d-8c67-4df2-8536-87579e706fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:13.576805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad20a0d-8c67-4df2-8536-87579e706fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:13.576805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad20a0d-8c67-4df2-8536-87579e706fd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:13.589662 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad20a0d-8c67-4df2-8536-87579e706fd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:13.592178 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad20a0d-8c67-4df2-8536-87579e706fd7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:13.603784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af094635-8893-4178-9a16-d641b4cd4294 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:13.612884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-8ad20a0d-8c67-4df2-8536-87579e706fd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:13.612884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:13.612884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:13.612884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:13.612884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:13.612884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:13.612884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:13.612884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:13.612884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:13.612884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:13.612884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:13.612884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:13.612884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:13.612884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:13.612884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:13.617967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:13.617967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:13.617967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:13.617967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:13.617967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:13.617967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:13.617967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:13.617967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:13.617967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:13.617967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:13.617967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:13.617967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:13.617967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:13.617967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:13.617967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:13.617967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:13.617967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:13.619987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:13.619987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:13.619987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:13.619987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:13.619987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:13.619987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:13.619987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:13.619987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:13.619987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 337/677] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:13.619987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad20a0d-8c67-4df2-8536-87579e706fd7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:13.627693 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:13.691436 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6b812-aa8f-46c6-a9f9-a94e26ae214a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:13.692058 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6b812-aa8f-46c6-a9f9-a94e26ae214a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:13.692333 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6b812-aa8f-46c6-a9f9-a94e26ae214a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:13.693549 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6b812-aa8f-46c6-a9f9-a94e26ae214a None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:13.695693 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6b812-aa8f-46c6-a9f9-a94e26ae214a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:13.697477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6b812-aa8f-46c6-a9f9-a94e26ae214a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:13.698975 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6b812-aa8f-46c6-a9f9-a94e26ae214a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:13.721804 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 341/678] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Sep 6 16:01:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:13.722708 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6b812-aa8f-46c6-a9f9-a94e26ae214a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:13.748088 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:13.767278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad20a0d-8c67-4df2-8536-87579e706fd7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:13.812354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce59e9e-a662-4006-a2bd-1047e298c892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:13.813005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce59e9e-a662-4006-a2bd-1047e298c892 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:13.813290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce59e9e-a662-4006-a2bd-1047e298c892 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:13.813565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce59e9e-a662-4006-a2bd-1047e298c892 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:13.814221 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce59e9e-a662-4006-a2bd-1047e298c892 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:13.814521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce59e9e-a662-4006-a2bd-1047e298c892 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:13.816010 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce59e9e-a662-4006-a2bd-1047e298c892 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:13.839633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 338/679] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:13.839633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec4898-e54b-4142-957c-e81116cbc848 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:13.839633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec4898-e54b-4142-957c-e81116cbc848 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:13.839633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec4898-e54b-4142-957c-e81116cbc848 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:13.839633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 339/680] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:13.841937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec4898-e54b-4142-957c-e81116cbc848 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:13.875288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6b812-aa8f-46c6-a9f9-a94e26ae214a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:13.919185 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b4213-789b-4f1d-b7e3-bb561823f470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:13.919943 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b4213-789b-4f1d-b7e3-bb561823f470 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:13.920326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b4213-789b-4f1d-b7e3-bb561823f470 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:13.920776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b4213-789b-4f1d-b7e3-bb561823f470 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:13.921607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b4213-789b-4f1d-b7e3-bb561823f470 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:13.921976 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b4213-789b-4f1d-b7e3-bb561823f470 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:13.923978 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b4213-789b-4f1d-b7e3-bb561823f470 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:13.940540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 342/681] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:13.943244 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778f5a5-a0dc-4ef9-80a8-c2433b7b2e5b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:13.943720 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778f5a5-a0dc-4ef9-80a8-c2433b7b2e5b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:13.944194 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778f5a5-a0dc-4ef9-80a8-c2433b7b2e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:13.968155 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b778f5a5-a0dc-4ef9-80a8-c2433b7b2e5b None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:13.969383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b778f5a5-a0dc-4ef9-80a8-c2433b7b2e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:13.973301 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec4898-e54b-4142-957c-e81116cbc848 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:14.011014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:14.021381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c2252-7276-474a-a62f-28da892b615f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:14.022742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c2252-7276-474a-a62f-28da892b615f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:14.023027 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c2252-7276-474a-a62f-28da892b615f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:14.023294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c2252-7276-474a-a62f-28da892b615f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:14.042623 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:14.103580 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 343/682] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:01:14.105092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778f5a5-a0dc-4ef9-80a8-c2433b7b2e5b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:14.108589 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c2252-7276-474a-a62f-28da892b615f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:14.109641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c2252-7276-474a-a62f-28da892b615f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:14.267682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778f5a5-a0dc-4ef9-80a8-c2433b7b2e5b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:14.307251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 340/683] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:01:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:01:14.308905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c2252-7276-474a-a62f-28da892b615f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:14.352405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ed51d-e838-44f3-9dc7-515f43751788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:14.354908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ed51d-e838-44f3-9dc7-515f43751788 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:14.356380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ed51d-e838-44f3-9dc7-515f43751788 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:14.357191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ed51d-e838-44f3-9dc7-515f43751788 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:14.375580 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ed51d-e838-44f3-9dc7-515f43751788 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:14.380536 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ed51d-e838-44f3-9dc7-515f43751788 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:14.390042 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:14.415366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-ea5ed51d-e838-44f3-9dc7-515f43751788 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:14.415366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:14.415366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:14.415366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:14.415366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:14.415366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:14.415366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:14.415366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:14.415366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:14.415366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:14.415366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:14.415366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:14.415366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:14.415366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:14.415366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:14.417101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:14.417101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:14.417101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:14.417101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:14.417101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:14.417101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:14.417101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:14.417101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:14.417101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:14.417101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:14.417101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:14.417101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:14.417101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:14.417101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:14.417101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:14.417101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:14.417101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:14.418778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:14.418778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:14.418778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:14.418778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:14.418778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:14.418778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:14.418778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:14.418778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:14.421070 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 344/684] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:14.421459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ed51d-e838-44f3-9dc7-515f43751788 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:14.456435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:14.487722 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c2252-7276-474a-a62f-28da892b615f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:14.575987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8efa04-5320-4b51-ae91-fb6589b98787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:14.577038 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8efa04-5320-4b51-ae91-fb6589b98787 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:14.577615 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8efa04-5320-4b51-ae91-fb6589b98787 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:14.578188 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8efa04-5320-4b51-ae91-fb6589b98787 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:14.579739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8efa04-5320-4b51-ae91-fb6589b98787 None admin] RBAC: Adding query filter params (role.id=56d621cf28924933b6356795a6f8ba23, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=989beaca146d4f9b950f333bd3567e6a) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:14.580400 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8efa04-5320-4b51-ae91-fb6589b98787 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:14.583972 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8efa04-5320-4b51-ae91-fb6589b98787 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:14.596412 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 341/685] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:01:14 2022] GET /identity/v3/role_assignments?user.id=989beaca146d4f9b950f333bd3567e6a&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=56d621cf28924933b6356795a6f8ba23 => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:14.597869 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8efa04-5320-4b51-ae91-fb6589b98787 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:14.621310 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ed51d-e838-44f3-9dc7-515f43751788 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:14.711347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dfccb9-562f-4c5b-953f-77e195f2faea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:14.712757 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfccb9-562f-4c5b-953f-77e195f2faea None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:14.713245 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfccb9-562f-4c5b-953f-77e195f2faea None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:14.713703 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfccb9-562f-4c5b-953f-77e195f2faea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:14.737989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dfccb9-562f-4c5b-953f-77e195f2faea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:14.742317 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dfccb9-562f-4c5b-953f-77e195f2faea None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:14.780632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-c3dfccb9-562f-4c5b-953f-77e195f2faea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:14.780632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:14.780632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:14.780632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:14.780632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:14.780632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:14.780632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:14.780632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:14.780632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:14.780632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:14.780632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:14.780632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:14.780632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:14.780632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:14.780632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:14.780632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:14.781727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:14.781727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:14.781727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:14.781727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:14.781727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:14.781727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:14.781727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:14.781727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:14.781727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:14.781727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:14.781727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:14.781727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:14.781727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:14.781727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:14.781727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:14.781727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:14.781727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:14.782729 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:14.782729 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:14.782729 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:14.790970 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 345/686] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:14.792566 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dfccb9-562f-4c5b-953f-77e195f2faea None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:14.821782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8efa04-5320-4b51-ae91-fb6589b98787 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:14.835706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:14.886004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3efd6-fa0d-4897-9d8a-41fee5f00834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:14.887758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3efd6-fa0d-4897-9d8a-41fee5f00834 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:14.887758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3efd6-fa0d-4897-9d8a-41fee5f00834 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:14.887758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3efd6-fa0d-4897-9d8a-41fee5f00834 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:14.888041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac3efd6-fa0d-4897-9d8a-41fee5f00834 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:14.888298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac3efd6-fa0d-4897-9d8a-41fee5f00834 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:14.889800 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac3efd6-fa0d-4897-9d8a-41fee5f00834 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:14.909148 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 342/687] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:14.911240 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3efd6-fa0d-4897-9d8a-41fee5f00834 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:14.940041 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:14.987532 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dfccb9-562f-4c5b-953f-77e195f2faea None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:15.064945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb5e86-3aa7-4e54-bafd-fc5e69fb5234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:15.066502 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb5e86-3aa7-4e54-bafd-fc5e69fb5234 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:15.067344 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb5e86-3aa7-4e54-bafd-fc5e69fb5234 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:15.068137 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb5e86-3aa7-4e54-bafd-fc5e69fb5234 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:15.083978 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb5e86-3aa7-4e54-bafd-fc5e69fb5234 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:15.086052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb5e86-3aa7-4e54-bafd-fc5e69fb5234 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:15.104959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-23eb5e86-3aa7-4e54-bafd-fc5e69fb5234 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:15.104959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:15.104959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:15.104959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:15.104959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:15.104959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:15.104959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:15.104959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:15.104959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:15.104959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:15.104959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:15.104959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:15.104959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:15.104959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:15.104959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:15.104959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:15.105964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:15.105964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:15.105964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:15.105964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:15.105964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:15.105964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:15.105964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:15.105964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:15.105964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:15.105964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:15.105964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:15.105964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:15.105964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:15.105964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:15.105964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:15.105964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:15.105964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:15.106879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:15.106879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:15.106879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:15.114075 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 346/688] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:15.116138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb5e86-3aa7-4e54-bafd-fc5e69fb5234 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:15.139667 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3efd6-fa0d-4897-9d8a-41fee5f00834 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:15.214783 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19247a-6d5f-4aef-bcf5-81160f85d12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:15.215326 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19247a-6d5f-4aef-bcf5-81160f85d12a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:15.215816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19247a-6d5f-4aef-bcf5-81160f85d12a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:15.216298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19247a-6d5f-4aef-bcf5-81160f85d12a None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:15.217506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19247a-6d5f-4aef-bcf5-81160f85d12a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:15.218050 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19247a-6d5f-4aef-bcf5-81160f85d12a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:15.222017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19247a-6d5f-4aef-bcf5-81160f85d12a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:15.247578 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 343/689] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:15.249255 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19247a-6d5f-4aef-bcf5-81160f85d12a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:15.257769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb5e86-3aa7-4e54-bafd-fc5e69fb5234 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:15.295261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:15.316139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:15.321478 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77008b7e-74e1-471d-b37c-8a7ae907b60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:15.322134 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77008b7e-74e1-471d-b37c-8a7ae907b60f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:15.322505 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77008b7e-74e1-471d-b37c-8a7ae907b60f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:15.322811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77008b7e-74e1-471d-b37c-8a7ae907b60f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:15.323485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77008b7e-74e1-471d-b37c-8a7ae907b60f None admin] RBAC: Adding query filter params (role.id=56d621cf28924933b6356795a6f8ba23, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=7849ef049ffd4495909f997735af8782) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:15.323795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77008b7e-74e1-471d-b37c-8a7ae907b60f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:15.326001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77008b7e-74e1-471d-b37c-8a7ae907b60f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:15.351126 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 347/690] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:01:15 2022] GET /identity/v3/role_assignments?user.id=7849ef049ffd4495909f997735af8782&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=56d621cf28924933b6356795a6f8ba23 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:15.351126 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77008b7e-74e1-471d-b37c-8a7ae907b60f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:15.553322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19247a-6d5f-4aef-bcf5-81160f85d12a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:15.623139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77008b7e-74e1-471d-b37c-8a7ae907b60f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:15.662958 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d061911-dc5b-4a58-a40e-0263acaf5f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:15.663911 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d061911-dc5b-4a58-a40e-0263acaf5f08 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:15.664372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d061911-dc5b-4a58-a40e-0263acaf5f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:15.664832 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d061911-dc5b-4a58-a40e-0263acaf5f08 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:15.665928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d061911-dc5b-4a58-a40e-0263acaf5f08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:15.666446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d061911-dc5b-4a58-a40e-0263acaf5f08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:15.668913 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d061911-dc5b-4a58-a40e-0263acaf5f08 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:15.697318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 344/691] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 449 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:15.705552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f302d-6457-46ee-ac68-1c4b04df893c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:15.706925 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f302d-6457-46ee-ac68-1c4b04df893c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:15.707548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d061911-dc5b-4a58-a40e-0263acaf5f08 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:15.707860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f302d-6457-46ee-ac68-1c4b04df893c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:15.709236 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f302d-6457-46ee-ac68-1c4b04df893c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:15.728778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f302d-6457-46ee-ac68-1c4b04df893c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:15.733143 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f302d-6457-46ee-ac68-1c4b04df893c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:15.764301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-257f302d-6457-46ee-ac68-1c4b04df893c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:15.764301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:15.764301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:15.764301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:15.764301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:15.764301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:15.764301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:15.764301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:15.764301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:15.764301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:15.764301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:15.764301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:15.764301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:15.764301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:15.764301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:15.766340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:15.766340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:15.766340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:15.766340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:15.766340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:15.766340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:15.766340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:15.766340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:15.766340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:15.766340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:15.766340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:15.766340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:15.766340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:15.766340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:15.766340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:15.766340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:15.766340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:15.768497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:15.768497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:15.768497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:15.768497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:15.768497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:15.768497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:15.768497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:15.768497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:15.772654 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 348/692] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 426 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:15.779672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f302d-6457-46ee-ac68-1c4b04df893c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:15.857599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:15.910739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:15.948289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d061911-dc5b-4a58-a40e-0263acaf5f08 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:16.001542 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f302d-6457-46ee-ac68-1c4b04df893c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:16.062648 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496324f-1231-4d96-aad4-f385be09bc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:16.063632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496324f-1231-4d96-aad4-f385be09bc10 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:16.064083 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496324f-1231-4d96-aad4-f385be09bc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:16.064521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496324f-1231-4d96-aad4-f385be09bc10 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:16.065805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496324f-1231-4d96-aad4-f385be09bc10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:16.069102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496324f-1231-4d96-aad4-f385be09bc10 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:16.091888 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74061455-4c6d-429b-ad3d-19051596c01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:16.092921 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74061455-4c6d-429b-ad3d-19051596c01b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:16.093406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74061455-4c6d-429b-ad3d-19051596c01b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:16.093850 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74061455-4c6d-429b-ad3d-19051596c01b None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:16.095180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74061455-4c6d-429b-ad3d-19051596c01b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:16.095704 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74061455-4c6d-429b-ad3d-19051596c01b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:16.098308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74061455-4c6d-429b-ad3d-19051596c01b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:16.124919 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 349/693] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:16.132144 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74061455-4c6d-429b-ad3d-19051596c01b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:16.134108 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 345/694] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 6 16:01:15 2022] POST /identity/v3/users => generated 254 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:01:16.337994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74061455-4c6d-429b-ad3d-19051596c01b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:16.353691 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:16.408749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142949c-6d3b-4c70-9102-8145906730ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:16.409629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142949c-6d3b-4c70-9102-8145906730ae None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:16.410103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142949c-6d3b-4c70-9102-8145906730ae None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:16.410570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142949c-6d3b-4c70-9102-8145906730ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:16.412489 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142949c-6d3b-4c70-9102-8145906730ae None admin] RBAC: Adding query filter params (role.id=56d621cf28924933b6356795a6f8ba23, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=ece176d412574790a13d03dcca08f91a) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:16.413227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142949c-6d3b-4c70-9102-8145906730ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:16.415985 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142949c-6d3b-4c70-9102-8145906730ae None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:16.428570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 350/695] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:01:16 2022] GET /identity/v3/role_assignments?user.id=ece176d412574790a13d03dcca08f91a&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=56d621cf28924933b6356795a6f8ba23 => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:16.596889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1b8df-bfe9-44f3-8b0c-3078b26e0e5a None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:16.607115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1b8df-bfe9-44f3-8b0c-3078b26e0e5a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:16.607850 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1b8df-bfe9-44f3-8b0c-3078b26e0e5a None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:16.615393 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 346/696] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:16 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:16.620823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b765b1a-80ca-4771-a0fc-bc6274cea382 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:16.621429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b765b1a-80ca-4771-a0fc-bc6274cea382 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:16.622006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b765b1a-80ca-4771-a0fc-bc6274cea382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:16.659188 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-3b765b1a-80ca-4771-a0fc-bc6274cea382 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:16.661189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3b765b1a-80ca-4771-a0fc-bc6274cea382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:16.806260 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 351/697] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 16:01:17.341610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1b8df-bfe9-44f3-8b0c-3078b26e0e5a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:17.497799 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12220216-b7f6-4eda-824a-d06c68054e82 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:17.498414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12220216-b7f6-4eda-824a-d06c68054e82 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:17.498781 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12220216-b7f6-4eda-824a-d06c68054e82 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:17.501696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 352/698] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:17.515764 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d838520-772a-4feb-950c-d6889c40cc8b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:17.516258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d838520-772a-4feb-950c-d6889c40cc8b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:17.516741 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d838520-772a-4feb-950c-d6889c40cc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:17.560217 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1b8df-bfe9-44f3-8b0c-3078b26e0e5a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:17.576422 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8d838520-772a-4feb-950c-d6889c40cc8b None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:17.578511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8d838520-772a-4feb-950c-d6889c40cc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:17.673811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6e2b2-34d7-4779-a499-22b0aea03998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:17.674842 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6e2b2-34d7-4779-a499-22b0aea03998 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:17.675565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6e2b2-34d7-4779-a499-22b0aea03998 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:17.676036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6e2b2-34d7-4779-a499-22b0aea03998 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:17.714608 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6e2b2-34d7-4779-a499-22b0aea03998 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:01:17.719575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-04e6e2b2-34d7-4779-a499-22b0aea03998 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:01:17.719575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:17.719575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:17.719575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:17.719575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:17.719575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:17.719575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:17.719575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:17.719575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:17.719575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:17.719575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:17.719575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:17.719575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:01:17.719575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:01:17.719575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:01:17.719575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:01:17.721429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:01:17.721429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:01:17.721429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:01:17.721429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:01:17.721429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:17.721429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:17.721429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:17.721429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:17.721429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:17.721429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:17.721429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:17.721429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:17.721429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:17.721429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:17.721429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:17.721429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:17.724489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:17.724489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:17.724489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:17.724489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:17.724489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:01:17.724489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:01:17.724489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:01:17.724489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:01:17.724489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:01:17.724489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:01:17.724489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:01:17.724489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:17.726611 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 347/699] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:01:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:17.739297 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6e2b2-34d7-4779-a499-22b0aea03998 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:17.750366 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:17.788386 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 353/700] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:01:17.983554 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6e2b2-34d7-4779-a499-22b0aea03998 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:18.076374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e3428-b644-4d96-bcb7-7bb949f8fc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:18.077950 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e3428-b644-4d96-bcb7-7bb949f8fc7c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:18.078790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e3428-b644-4d96-bcb7-7bb949f8fc7c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:18.079568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e3428-b644-4d96-bcb7-7bb949f8fc7c None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:18.081114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e3428-b644-4d96-bcb7-7bb949f8fc7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:18.081978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e3428-b644-4d96-bcb7-7bb949f8fc7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:18.086505 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e3428-b644-4d96-bcb7-7bb949f8fc7c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:18.111175 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 348/701] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:01:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:18.116162 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d838520-772a-4feb-950c-d6889c40cc8b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:18.187158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:18.287976 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e3428-b644-4d96-bcb7-7bb949f8fc7c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:18.318926 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d838520-772a-4feb-950c-d6889c40cc8b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:18.413064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10b505-61c8-408f-b6fc-9e3c20ac3fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:18.413746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10b505-61c8-408f-b6fc-9e3c20ac3fcb None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:18.414019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10b505-61c8-408f-b6fc-9e3c20ac3fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:18.414291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10b505-61c8-408f-b6fc-9e3c20ac3fcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:18.434920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10b505-61c8-408f-b6fc-9e3c20ac3fcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:18.438652 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:18.440787 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10b505-61c8-408f-b6fc-9e3c20ac3fcb None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:18.461890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-cd10b505-61c8-408f-b6fc-9e3c20ac3fcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:18.461890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:18.461890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:18.461890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:18.461890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:18.461890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:18.461890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:18.461890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:18.461890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:18.461890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:18.461890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:18.461890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:18.461890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:18.461890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:18.461890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:18.461890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:18.463734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:18.463734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:18.463734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:18.463734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:18.463734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:18.463734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:18.463734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:18.463734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:18.463734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:18.463734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:18.463734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:18.463734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:18.463734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:18.463734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:18.463734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:18.463734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:18.463734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:18.465611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:18.465611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:18.465611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:18.465611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 354/702] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:18.465611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f80a2-0f5f-46f5-b46a-59f236bccdae None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:18.465611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f80a2-0f5f-46f5-b46a-59f236bccdae None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:18.465611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f80a2-0f5f-46f5-b46a-59f236bccdae None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:18.473429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 355/703] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:18.476857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a2588-0126-4f57-9db4-6465f27c4f50 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:18.477326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a2588-0126-4f57-9db4-6465f27c4f50 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:18.477806 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a2588-0126-4f57-9db4-6465f27c4f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:18.484272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e3428-b644-4d96-bcb7-7bb949f8fc7c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:18.508416 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f62a2588-0126-4f57-9db4-6465f27c4f50 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:18.510567 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f62a2588-0126-4f57-9db4-6465f27c4f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:18.526588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a2d05-a6ee-412a-bd36-4ddc03ca0ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:18.527275 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a2d05-a6ee-412a-bd36-4ddc03ca0ace None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:18.527534 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a2d05-a6ee-412a-bd36-4ddc03ca0ace None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:18.527812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a2d05-a6ee-412a-bd36-4ddc03ca0ace None admin] PATH_INFO: `/v3/services/network` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:18.528416 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a2d05-a6ee-412a-bd36-4ddc03ca0ace None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:18.530332 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a2d05-a6ee-412a-bd36-4ddc03ca0ace None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:18.545888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-185a2d05-a6ee-412a-bd36-4ddc03ca0ace None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 16:01:18.545888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:18.545888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:01:18.545888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:01:18.545888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:01:18.545888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:01:18.545888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:01:18.545888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:18.545888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 16:01:18.545888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:18.545888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:01:18.545888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:18.545888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:18.545888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:18.545888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:18.545888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:18.545888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:18.549630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:18.549630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:18.549630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:18.549630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:18.549630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:18.549630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:18.549630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:01:18.549630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:01:18.549630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:01:18.549630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:01:18.549630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:18.549630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:18.549630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:18.549630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:18.549630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:18.549630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:18.553190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:18.553190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:18.553190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:18.553190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:18.553190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:18.553190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:18.553190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:18.553190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:18.553190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:18.553190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:18.553190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:01:18.553190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:18.553190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 16:01:18.553190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:18.553190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 349/704] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:18 2022] GET /identity/v3/services/network => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:18.575368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a2d05-a6ee-412a-bd36-4ddc03ca0ace None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:18.631123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 356/705] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:01:18.632100 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a2588-0126-4f57-9db4-6465f27c4f50 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:18.648455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:18.769956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a2588-0126-4f57-9db4-6465f27c4f50 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:18.779110 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a2d05-a6ee-412a-bd36-4ddc03ca0ace None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:18.875424 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:18.888351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8de3e6-b10b-4b19-a147-847df62b2492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:18.889469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8de3e6-b10b-4b19-a147-847df62b2492 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:18.889956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8de3e6-b10b-4b19-a147-847df62b2492 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:18.890331 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8de3e6-b10b-4b19-a147-847df62b2492 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:18.891588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8de3e6-b10b-4b19-a147-847df62b2492 None admin] RBAC: Adding query filter params (name=network) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:18.892141 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8de3e6-b10b-4b19-a147-847df62b2492 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:18.894962 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8de3e6-b10b-4b19-a147-847df62b2492 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:18.901144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a91c6c-ac37-4fd7-8eca-f2d54e21840d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:18.902201 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a91c6c-ac37-4fd7-8eca-f2d54e21840d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:18.902800 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a91c6c-ac37-4fd7-8eca-f2d54e21840d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:18.903304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a91c6c-ac37-4fd7-8eca-f2d54e21840d None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:18.904509 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a91c6c-ac37-4fd7-8eca-f2d54e21840d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:18.905054 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a91c6c-ac37-4fd7-8eca-f2d54e21840d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:18.908118 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a91c6c-ac37-4fd7-8eca-f2d54e21840d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:18.922165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 357/706] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 6 16:01:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:18.932887 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8de3e6-b10b-4b19-a147-847df62b2492 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:18.949064 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 350/707] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:18.959026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a91c6c-ac37-4fd7-8eca-f2d54e21840d None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:19.209469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:19.259036 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8de3e6-b10b-4b19-a147-847df62b2492 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:19.262163 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a91c6c-ac37-4fd7-8eca-f2d54e21840d None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:19.298736 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7dae4-d63a-42b4-b02e-65a842559057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:19.299895 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7dae4-d63a-42b4-b02e-65a842559057 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:19.299895 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7dae4-d63a-42b4-b02e-65a842559057 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:19.299895 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7dae4-d63a-42b4-b02e-65a842559057 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:19.300748 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7dae4-d63a-42b4-b02e-65a842559057 None admin] RBAC: Adding query filter params (type=network) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:19.301054 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7dae4-d63a-42b4-b02e-65a842559057 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:19.302626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7dae4-d63a-42b4-b02e-65a842559057 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:19.310240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 358/708] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 6 16:01:18 2022] GET /identity/v3/services?type=network => generated 124 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:19.316996 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7dae4-d63a-42b4-b02e-65a842559057 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:19.318368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7877245-21e2-4ee0-abcf-bd130d4c3736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:19.318835 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7877245-21e2-4ee0-abcf-bd130d4c3736 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:19.318974 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7877245-21e2-4ee0-abcf-bd130d4c3736 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:19.319257 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7877245-21e2-4ee0-abcf-bd130d4c3736 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:19.330420 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7877245-21e2-4ee0-abcf-bd130d4c3736 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:19.332031 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7877245-21e2-4ee0-abcf-bd130d4c3736 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:19.346265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-c7877245-21e2-4ee0-abcf-bd130d4c3736 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:01:19.346265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:19.346265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:19.346265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:19.346265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:19.346265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:19.346265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:19.346265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:19.346265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:19.346265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:19.346265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:19.346265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:19.346265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:19.346265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:19.346265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:19.346265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:19.348127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:19.348127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:19.348127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:19.348127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:19.348127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:19.348127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:19.348127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:19.348127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:19.348127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:19.348127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:19.348127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:19.348127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:19.348127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:19.348127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:19.348127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:19.348127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:19.348127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:19.348127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:19.351581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:19.351581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:19.351581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:19.351581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:19.351581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:19.351581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:19.351581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:19.351581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:19.351581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:01:19.351581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:19.352998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 351/709] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Sep 6 16:01:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:19.355068 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5f5fb-8d45-489c-a674-9fd2b7fee59d None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:19.355068 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5f5fb-8d45-489c-a674-9fd2b7fee59d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:19.355608 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5f5fb-8d45-489c-a674-9fd2b7fee59d None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:19.356885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 352/710] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:19.357721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5f5fb-8d45-489c-a674-9fd2b7fee59d None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:19.436235 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:19.507262 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5f5fb-8d45-489c-a674-9fd2b7fee59d None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:19.562830 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7dae4-d63a-42b4-b02e-65a842559057 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:19.567414 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b00cf6-9f94-4084-8f18-f5105ec5b5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:19.568083 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b00cf6-9f94-4084-8f18-f5105ec5b5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:19.568371 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b00cf6-9f94-4084-8f18-f5105ec5b5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:19.568636 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b00cf6-9f94-4084-8f18-f5105ec5b5e9 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:19.569390 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b00cf6-9f94-4084-8f18-f5105ec5b5e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:19.569701 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b00cf6-9f94-4084-8f18-f5105ec5b5e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:19.571267 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b00cf6-9f94-4084-8f18-f5105ec5b5e9 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:19.595989 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 353/711] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Sep 6 16:01:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:19.597860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52daa96-7f47-4a4e-a4da-29cdab41ebf2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:19.598381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52daa96-7f47-4a4e-a4da-29cdab41ebf2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:19.599565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52daa96-7f47-4a4e-a4da-29cdab41ebf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:19.621476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b548f-bf17-4612-ae72-b19cdb389879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:19.622193 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b548f-bf17-4612-ae72-b19cdb389879 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:19.622479 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b548f-bf17-4612-ae72-b19cdb389879 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:19.623412 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b548f-bf17-4612-ae72-b19cdb389879 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:19.623911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b548f-bf17-4612-ae72-b19cdb389879 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:19.625381 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b548f-bf17-4612-ae72-b19cdb389879 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:19.628474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e52daa96-7f47-4a4e-a4da-29cdab41ebf2 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:19.630565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e52daa96-7f47-4a4e-a4da-29cdab41ebf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:19.640499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-249b548f-bf17-4612-ae72-b19cdb389879 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 16:01:19.640499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:19.640499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:01:19.640499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:01:19.640499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:01:19.640499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:01:19.640499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:01:19.640499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:19.640499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 16:01:19.640499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:19.640499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:01:19.640499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:19.640499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:19.640499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:19.640499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:19.640499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:19.640499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:19.641608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:19.641608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:19.641608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:19.641608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:19.641608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:19.641608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:19.641608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:01:19.641608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:01:19.641608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:01:19.641608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:01:19.641608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:19.641608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:19.641608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:19.641608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:19.641608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:19.641608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:19.643019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:19.643019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:19.643019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:19.643019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:19.643019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:19.643019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:19.643019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:19.643019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:19.643019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:19.643019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:19.643019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:01:19.643019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:19.643019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 16:01:19.643019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:19.646763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 359/712] 10.222.0.26 () {56 vars in 1129 bytes} [Tue Sep 6 16:01:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:19.650027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b548f-bf17-4612-ae72-b19cdb389879 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:19.657269 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:19.736042 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 354/713] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:01:19.737889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52daa96-7f47-4a4e-a4da-29cdab41ebf2 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:19.839566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:19.854687 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b548f-bf17-4612-ae72-b19cdb389879 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:19.924020 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a201f7-341d-41bd-a41d-e6bb4cdf5104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:19.926495 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a201f7-341d-41bd-a41d-e6bb4cdf5104 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:19.927494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a201f7-341d-41bd-a41d-e6bb4cdf5104 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:19.927870 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a201f7-341d-41bd-a41d-e6bb4cdf5104 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:20.010610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52daa96-7f47-4a4e-a4da-29cdab41ebf2 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:20.076214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a201f7-341d-41bd-a41d-e6bb4cdf5104 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:20.078904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a201f7-341d-41bd-a41d-e6bb4cdf5104 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:20.084345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:20.129032 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd74e44-600a-4dd8-995c-4c8b1e607625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:20.130039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd74e44-600a-4dd8-995c-4c8b1e607625 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:20.130547 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd74e44-600a-4dd8-995c-4c8b1e607625 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:20.131088 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd74e44-600a-4dd8-995c-4c8b1e607625 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:20.132298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd74e44-600a-4dd8-995c-4c8b1e607625 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:20.132859 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd74e44-600a-4dd8-995c-4c8b1e607625 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:20.135655 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd74e44-600a-4dd8-995c-4c8b1e607625 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:20.144757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 355/714] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Sep 6 16:01:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:20.146105 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd74e44-600a-4dd8-995c-4c8b1e607625 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:20.269173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 360/715] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:01:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:01:20.271317 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a201f7-341d-41bd-a41d-e6bb4cdf5104 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:20.277853 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd74e44-600a-4dd8-995c-4c8b1e607625 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:20.279805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:20.323212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae3fbd-ecc0-4b01-8f7d-761ff052637e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:20.324239 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae3fbd-ecc0-4b01-8f7d-761ff052637e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:20.324817 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae3fbd-ecc0-4b01-8f7d-761ff052637e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:20.325418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae3fbd-ecc0-4b01-8f7d-761ff052637e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:20.341258 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae3fbd-ecc0-4b01-8f7d-761ff052637e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:01:20.347863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-84ae3fbd-ecc0-4b01-8f7d-761ff052637e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:01:20.347863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:20.347863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:20.347863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:20.347863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:20.347863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:20.347863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:20.347863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:20.347863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:20.347863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:20.347863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:20.347863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:20.347863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:01:20.347863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:01:20.347863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:01:20.347863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:01:20.349246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:01:20.349246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:01:20.349246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:01:20.349246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:01:20.349246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:20.349246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:20.349246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:20.349246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:20.349246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:20.349246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:20.349246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:20.349246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:20.349246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:20.349246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:20.349246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:20.349246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:20.350382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:20.350382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:20.350382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:20.350382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:20.350382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:01:20.350382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:01:20.350382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:01:20.350382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:01:20.350382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:01:20.350382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:01:20.350382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:01:20.350382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:20.354883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 356/716] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:01:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:20.356832 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae3fbd-ecc0-4b01-8f7d-761ff052637e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:20.423493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a201f7-341d-41bd-a41d-e6bb4cdf5104 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:20.429456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:20.487984 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf77dbe-dbf3-4bb9-ac8c-94ad2894949b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:20.489025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf77dbe-dbf3-4bb9-ac8c-94ad2894949b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:20.489998 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf77dbe-dbf3-4bb9-ac8c-94ad2894949b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:20.491369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf77dbe-dbf3-4bb9-ac8c-94ad2894949b None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:20.492433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf77dbe-dbf3-4bb9-ac8c-94ad2894949b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:20.492913 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf77dbe-dbf3-4bb9-ac8c-94ad2894949b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:20.493437 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae3fbd-ecc0-4b01-8f7d-761ff052637e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:20.495229 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf77dbe-dbf3-4bb9-ac8c-94ad2894949b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:20.503734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 361/717] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Sep 6 16:01:20 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:20.507071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa8a38-820a-4e26-83a6-fa15111844a5 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:20.507578 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa8a38-820a-4e26-83a6-fa15111844a5 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:20.508000 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa8a38-820a-4e26-83a6-fa15111844a5 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:20.514852 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 362/718] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:20.514852 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa8a38-820a-4e26-83a6-fa15111844a5 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:20.553911 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd13ed-988b-4e24-9474-bb4201afa3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:20.555226 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd13ed-988b-4e24-9474-bb4201afa3bf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:20.555858 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd13ed-988b-4e24-9474-bb4201afa3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:20.556335 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd13ed-988b-4e24-9474-bb4201afa3bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:20.579083 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd13ed-988b-4e24-9474-bb4201afa3bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:20.582121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd13ed-988b-4e24-9474-bb4201afa3bf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:20.611327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-dffd13ed-988b-4e24-9474-bb4201afa3bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:20.611327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:20.611327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:20.611327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:20.611327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:20.611327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:20.611327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:20.611327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:20.611327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:20.611327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:20.611327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:20.611327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:20.611327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:20.611327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:20.611327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:20.611327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:20.612369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:20.612369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:20.612369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:20.612369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:20.612369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:20.612369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:20.612369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:20.612369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:20.612369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:20.612369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:20.612369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:20.612369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:20.612369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:20.612369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:20.612369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:20.612369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:20.612369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:20.613267 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:20.613267 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:20.613267 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:20.616457 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 357/719] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:20.618246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c05b5-a3a8-4792-8412-eef1827918b0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:20.618912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c05b5-a3a8-4792-8412-eef1827918b0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:20.619203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c05b5-a3a8-4792-8412-eef1827918b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:20.641804 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c39c05b5-a3a8-4792-8412-eef1827918b0 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:20.643173 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c39c05b5-a3a8-4792-8412-eef1827918b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:20.663820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:20.701416 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa8a38-820a-4e26-83a6-fa15111844a5 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:20.766698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b664f5-73fe-48fd-8016-58b304c724ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:20.766698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b664f5-73fe-48fd-8016-58b304c724ac None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:20.766698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b664f5-73fe-48fd-8016-58b304c724ac None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:20.766698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b664f5-73fe-48fd-8016-58b304c724ac None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:20.767095 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b664f5-73fe-48fd-8016-58b304c724ac None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:20.767095 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b664f5-73fe-48fd-8016-58b304c724ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:20.767095 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b664f5-73fe-48fd-8016-58b304c724ac None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:20.774357 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 358/720] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:01:20.775098 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 363/721] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:01:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:20.775825 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c05b5-a3a8-4792-8412-eef1827918b0 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:20.780824 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b664f5-73fe-48fd-8016-58b304c724ac None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:20.857336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:20.993658 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c05b5-a3a8-4792-8412-eef1827918b0 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:21.006194 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b664f5-73fe-48fd-8016-58b304c724ac None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:21.075152 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19582eb2-6749-4c81-b0d3-f3973eeda5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:21.076645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19582eb2-6749-4c81-b0d3-f3973eeda5fa None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:21.077347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19582eb2-6749-4c81-b0d3-f3973eeda5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:21.078043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19582eb2-6749-4c81-b0d3-f3973eeda5fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:21.097535 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19582eb2-6749-4c81-b0d3-f3973eeda5fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:21.100508 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19582eb2-6749-4c81-b0d3-f3973eeda5fa None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:21.108230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34af72-6f58-493e-a401-3633b68d703c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:21.109343 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34af72-6f58-493e-a401-3633b68d703c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:21.109836 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34af72-6f58-493e-a401-3633b68d703c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:21.110315 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34af72-6f58-493e-a401-3633b68d703c None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:21.111250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34af72-6f58-493e-a401-3633b68d703c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:21.112448 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34af72-6f58-493e-a401-3633b68d703c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:21.114143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34af72-6f58-493e-a401-3633b68d703c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:21.126558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-19582eb2-6749-4c81-b0d3-f3973eeda5fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:21.126558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:21.126558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:21.126558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:21.126558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:21.126558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:21.126558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:21.126558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:21.126558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:21.126558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:21.126558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:21.126558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:21.126558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:21.126558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:21.126558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:21.126558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:21.127634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:21.127634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:21.127634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:21.127634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:21.127634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:21.127634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:21.127634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:21.127634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:21.127634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:21.127634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:21.127634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:21.127634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:21.127634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:21.127634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:21.127634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:21.127634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:21.127634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:21.128501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:21.128501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:21.128501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:21.129162 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 364/722] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:21.130706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19582eb2-6749-4c81-b0d3-f3973eeda5fa None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:21.133059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 359/723] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:21.142850 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34af72-6f58-493e-a401-3633b68d703c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:21.148984 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:21.241897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:21.267916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19582eb2-6749-4c81-b0d3-f3973eeda5fa None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:21.323607 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34af72-6f58-493e-a401-3633b68d703c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:21.327626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870648bc-86c9-42f3-9f75-3980a7560163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:21.329208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870648bc-86c9-42f3-9f75-3980a7560163 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:21.329978 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870648bc-86c9-42f3-9f75-3980a7560163 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:21.330827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870648bc-86c9-42f3-9f75-3980a7560163 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:21.347462 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870648bc-86c9-42f3-9f75-3980a7560163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:01:21.355071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-870648bc-86c9-42f3-9f75-3980a7560163 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:01:21.355071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:21.355071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:21.355071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:21.355071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:21.355071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:21.355071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:21.355071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:21.355071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:21.355071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:21.355071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:21.355071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:21.355071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:01:21.355071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:01:21.355071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:01:21.355071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:01:21.356828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:01:21.356828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:01:21.356828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:01:21.356828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:01:21.356828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:21.356828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:21.356828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:21.356828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:21.356828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:21.356828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:21.356828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:21.356828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:21.356828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:21.356828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:21.356828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:21.356828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:21.358484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:21.358484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:21.358484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:21.358484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:21.358484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:01:21.358484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:01:21.358484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:01:21.358484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:01:21.358484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:01:21.358484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:01:21.358484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:01:21.358484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:21.361418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 365/724] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:01:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:21.362115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870648bc-86c9-42f3-9f75-3980a7560163 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:21.397794 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc51154-9d40-4345-82e4-96ccc54bdd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:21.399488 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc51154-9d40-4345-82e4-96ccc54bdd26 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:21.400230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc51154-9d40-4345-82e4-96ccc54bdd26 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:21.401007 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc51154-9d40-4345-82e4-96ccc54bdd26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:21.415548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc51154-9d40-4345-82e4-96ccc54bdd26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:21.418870 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc51154-9d40-4345-82e4-96ccc54bdd26 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:21.442039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-4fc51154-9d40-4345-82e4-96ccc54bdd26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:21.442039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:21.442039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:21.442039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:21.442039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:21.442039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:21.442039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:21.442039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:21.442039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:21.442039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:21.442039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:21.442039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:21.442039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:21.442039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:21.442039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:21.445035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:21.445035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:21.445035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:21.445035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:21.445035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:21.445035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:21.445035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:21.445035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:21.445035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:21.445035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:21.445035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:21.445035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:21.445035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:21.445035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:21.445035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:21.445035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:21.445035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:21.448336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:21.448336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:21.448336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:21.448336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:21.448336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:21.448336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:21.448336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:21.448336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:21.452890 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 360/725] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:21.454694 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc51154-9d40-4345-82e4-96ccc54bdd26 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:21.554726 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870648bc-86c9-42f3-9f75-3980a7560163 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:21.591198 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:21.615493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546c599-5fdd-4613-b2af-7b25e4a3a94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:21.616462 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546c599-5fdd-4613-b2af-7b25e4a3a94e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:21.616937 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546c599-5fdd-4613-b2af-7b25e4a3a94e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:21.617400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546c599-5fdd-4613-b2af-7b25e4a3a94e None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:21.618540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546c599-5fdd-4613-b2af-7b25e4a3a94e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:21.619087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546c599-5fdd-4613-b2af-7b25e4a3a94e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:21.621598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546c599-5fdd-4613-b2af-7b25e4a3a94e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:21.650796 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 366/726] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:21.652354 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546c599-5fdd-4613-b2af-7b25e4a3a94e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:21.657017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:21.696273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc51154-9d40-4345-82e4-96ccc54bdd26 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:21.790361 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19e2ef-35b2-44b4-988f-e47a27c032da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:21.792466 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19e2ef-35b2-44b4-988f-e47a27c032da None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:21.793517 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19e2ef-35b2-44b4-988f-e47a27c032da None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:21.794452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19e2ef-35b2-44b4-988f-e47a27c032da None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:21.796838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19e2ef-35b2-44b4-988f-e47a27c032da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:21.797722 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19e2ef-35b2-44b4-988f-e47a27c032da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:21.798466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546c599-5fdd-4613-b2af-7b25e4a3a94e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:21.804447 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19e2ef-35b2-44b4-988f-e47a27c032da None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:21.828509 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 361/727] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:01:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:21.829384 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19e2ef-35b2-44b4-988f-e47a27c032da None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:21.877378 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6040e17-852e-4007-ad20-436a74f6e3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:21.878924 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6040e17-852e-4007-ad20-436a74f6e3c5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:21.879848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6040e17-852e-4007-ad20-436a74f6e3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:21.880806 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6040e17-852e-4007-ad20-436a74f6e3c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:21.883120 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6040e17-852e-4007-ad20-436a74f6e3c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:21.883120 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6040e17-852e-4007-ad20-436a74f6e3c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:21.884576 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6040e17-852e-4007-ad20-436a74f6e3c5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:21.899418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 367/728] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:21.900858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6040e17-852e-4007-ad20-436a74f6e3c5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:21.997970 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:22.031842 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19e2ef-35b2-44b4-988f-e47a27c032da None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:22.076589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6040e17-852e-4007-ad20-436a74f6e3c5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:22.098610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:22.112480 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f30bb-d36c-432a-bbc2-896bb0ea480c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:22.113300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f30bb-d36c-432a-bbc2-896bb0ea480c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:22.113701 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f30bb-d36c-432a-bbc2-896bb0ea480c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:22.114122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f30bb-d36c-432a-bbc2-896bb0ea480c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:22.124199 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f30bb-d36c-432a-bbc2-896bb0ea480c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:22.126287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f30bb-d36c-432a-bbc2-896bb0ea480c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:22.140964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-2c4f30bb-d36c-432a-bbc2-896bb0ea480c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:01:22.140964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:22.140964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:22.140964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:22.140964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:22.140964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:22.140964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:22.140964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:22.140964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:22.140964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:22.140964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:22.140964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:22.140964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:22.140964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:22.140964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:22.140964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:22.143398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:22.143398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:22.143398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:22.143398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:22.143398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:22.143398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:22.143398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:22.143398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:22.143398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:22.143398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:22.143398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:22.143398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:22.143398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:22.143398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:22.143398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:22.143398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:22.143398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:22.143398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:22.145568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:22.145568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:22.145568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:22.145568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:22.145568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:22.145568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:22.145568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:22.145568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:22.145568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:01:22.145568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:22.145568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 362/729] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 6 16:01:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:22.145568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f30bb-d36c-432a-bbc2-896bb0ea480c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:22.154165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b0637-7c0e-4de8-9ce0-269f0159d4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:22.155281 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b0637-7c0e-4de8-9ce0-269f0159d4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:22.155785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b0637-7c0e-4de8-9ce0-269f0159d4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:22.156269 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b0637-7c0e-4de8-9ce0-269f0159d4b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:22.178383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b0637-7c0e-4de8-9ce0-269f0159d4b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:22.181226 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b0637-7c0e-4de8-9ce0-269f0159d4b9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:22.209718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-da7b0637-7c0e-4de8-9ce0-269f0159d4b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:22.209718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:22.209718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:22.209718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:22.209718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:22.209718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:22.209718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:22.209718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:22.209718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:22.209718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:22.209718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:22.209718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:22.209718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:22.209718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:22.209718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:22.209718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:22.210783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:22.210783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:22.210783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:22.210783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:22.210783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:22.210783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:22.210783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:22.210783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:22.210783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:22.210783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:22.210783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:22.210783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:22.210783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:22.210783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:22.210783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:22.210783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:22.210783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:22.211689 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:22.211689 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:22.211689 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:22.211858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 368/730] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:22.214067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1eb69-1de0-4ffa-a596-23a7d1d7109d None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:22.214557 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1eb69-1de0-4ffa-a596-23a7d1d7109d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:22.215068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1eb69-1de0-4ffa-a596-23a7d1d7109d None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:22.221537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 369/731] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:22.221537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1eb69-1de0-4ffa-a596-23a7d1d7109d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:22.332243 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f30bb-d36c-432a-bbc2-896bb0ea480c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:22.387336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8005c5-13c3-4891-8d27-6bd44dade907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:22.388559 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8005c5-13c3-4891-8d27-6bd44dade907 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:22.388973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8005c5-13c3-4891-8d27-6bd44dade907 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:22.389368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8005c5-13c3-4891-8d27-6bd44dade907 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:22.390245 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8005c5-13c3-4891-8d27-6bd44dade907 None admin] RBAC: Adding query filter params (role.id=ba05677650e046a48d5f19440d48fbf2, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=6afd4f6ffdae4b16aa5ba757e68c18c4) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:22.391325 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8005c5-13c3-4891-8d27-6bd44dade907 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:22.392739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8005c5-13c3-4891-8d27-6bd44dade907 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:22.406694 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 363/732] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:01:22 2022] GET /identity/v3/role_assignments?user.id=6afd4f6ffdae4b16aa5ba757e68c18c4&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:22.406694 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8005c5-13c3-4891-8d27-6bd44dade907 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:22.423253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1eb69-1de0-4ffa-a596-23a7d1d7109d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:22.456067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:22.505648 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73f45f-4f30-4201-960e-34dba6c39795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:22.506912 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73f45f-4f30-4201-960e-34dba6c39795 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:22.507555 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73f45f-4f30-4201-960e-34dba6c39795 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:22.508117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73f45f-4f30-4201-960e-34dba6c39795 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:22.509784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73f45f-4f30-4201-960e-34dba6c39795 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:22.510514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73f45f-4f30-4201-960e-34dba6c39795 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:22.513760 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73f45f-4f30-4201-960e-34dba6c39795 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:22.518217 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:22.557988 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 370/733] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Sep 6 16:01:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:22.560782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235cb96-9310-451d-ae34-1e2e3efe5532 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:22.561611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235cb96-9310-451d-ae34-1e2e3efe5532 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:22.562378 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235cb96-9310-451d-ae34-1e2e3efe5532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:22.597997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f235cb96-9310-451d-ae34-1e2e3efe5532 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:22.600786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f235cb96-9310-451d-ae34-1e2e3efe5532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:22.663016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8005c5-13c3-4891-8d27-6bd44dade907 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:22.734225 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2d0e9-0274-4c22-80f7-366c4d0f507e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:22.735916 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2d0e9-0274-4c22-80f7-366c4d0f507e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:22.736687 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2d0e9-0274-4c22-80f7-366c4d0f507e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:22.737541 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2d0e9-0274-4c22-80f7-366c4d0f507e None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:22.739228 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2d0e9-0274-4c22-80f7-366c4d0f507e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:22.740591 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2d0e9-0274-4c22-80f7-366c4d0f507e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:22.743870 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2d0e9-0274-4c22-80f7-366c4d0f507e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:22.758551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 371/734] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:01:22.761359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235cb96-9310-451d-ae34-1e2e3efe5532 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:22.770439 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 364/735] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:22.778843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2d0e9-0274-4c22-80f7-366c4d0f507e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:22.955277 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235cb96-9310-451d-ae34-1e2e3efe5532 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:22.956572 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2d0e9-0274-4c22-80f7-366c4d0f507e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:22.957311 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:22.982889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:22.996646 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad28d0-81d4-41c0-8938-99395a033c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:22.997527 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad28d0-81d4-41c0-8938-99395a033c8d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:22.997951 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad28d0-81d4-41c0-8938-99395a033c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:22.998364 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad28d0-81d4-41c0-8938-99395a033c8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:23.035278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f257de2-5632-4104-8992-d7e16f66a295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:23.036882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f257de2-5632-4104-8992-d7e16f66a295 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:23.037374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f257de2-5632-4104-8992-d7e16f66a295 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:23.037870 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f257de2-5632-4104-8992-d7e16f66a295 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:23.053784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f257de2-5632-4104-8992-d7e16f66a295 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:23.056807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f257de2-5632-4104-8992-d7e16f66a295 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:23.075222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad28d0-81d4-41c0-8938-99395a033c8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:23.076686 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad28d0-81d4-41c0-8938-99395a033c8d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:23.081963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-3f257de2-5632-4104-8992-d7e16f66a295 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:01:23.081963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:23.081963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:23.081963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:23.081963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:23.081963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:23.081963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:23.081963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:23.081963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:23.081963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:23.081963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:23.081963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:23.081963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:23.081963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:23.081963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:23.081963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:23.083048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:23.083048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:23.083048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:23.083048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:23.083048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:23.083048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:23.083048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:23.083048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:23.083048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:23.083048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:23.083048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:23.083048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:23.083048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:23.083048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:23.083048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:23.083048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:23.083048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:23.083048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:23.084024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:23.084024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:23.084024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:23.084024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:23.084024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:23.084024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:23.084024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:23.084024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:23.084024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:01:23.084024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:23.085470 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 365/736] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:01:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:23.087979 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3066eb-1ad0-42bd-a242-e6fbc262fb5d None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:23.088551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3066eb-1ad0-42bd-a242-e6fbc262fb5d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:23.089061 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3066eb-1ad0-42bd-a242-e6fbc262fb5d None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:23.095670 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 366/737] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:23.095670 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3066eb-1ad0-42bd-a242-e6fbc262fb5d None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:23.233299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 372/738] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:01:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:01:23.234196 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad28d0-81d4-41c0-8938-99395a033c8d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:23.244046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:23.267538 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3066eb-1ad0-42bd-a242-e6fbc262fb5d None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:23.342566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094edc94-d733-4053-bd74-1c6d891a262e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:23.343619 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094edc94-d733-4053-bd74-1c6d891a262e None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:23.344127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094edc94-d733-4053-bd74-1c6d891a262e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:23.344643 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094edc94-d733-4053-bd74-1c6d891a262e None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:23.345831 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094edc94-d733-4053-bd74-1c6d891a262e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:23.348499 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094edc94-d733-4053-bd74-1c6d891a262e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:23.377688 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 367/739] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 6 16:01:23 2022] POST /identity/v3/services => generated 240 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:01:23.379811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81374f-fd53-4e65-a4f5-35f745e1081f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:23.380388 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81374f-fd53-4e65-a4f5-35f745e1081f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:23.380982 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81374f-fd53-4e65-a4f5-35f745e1081f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:23.385464 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad28d0-81d4-41c0-8938-99395a033c8d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:23.422514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-bc81374f-fd53-4e65-a4f5-35f745e1081f None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:23.425033 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bc81374f-fd53-4e65-a4f5-35f745e1081f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:23.467698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424c628-dedf-4873-a226-854e88b792f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:23.469107 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424c628-dedf-4873-a226-854e88b792f6 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:23.469609 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424c628-dedf-4873-a226-854e88b792f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:23.470082 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424c628-dedf-4873-a226-854e88b792f6 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:23.471124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:23.471496 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6424c628-dedf-4873-a226-854e88b792f6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:23.471822 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6424c628-dedf-4873-a226-854e88b792f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:23.474149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6424c628-dedf-4873-a226-854e88b792f6 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:23.510314 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 373/740] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Sep 6 16:01:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:23.511984 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424c628-dedf-4873-a226-854e88b792f6 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:23.556267 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 368/741] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:23 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 16:01:23.562715 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81374f-fd53-4e65-a4f5-35f745e1081f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:23.602600 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:23.667348 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424c628-dedf-4873-a226-854e88b792f6 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:23.736611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd023575-d8fe-4d5c-920a-83e528304bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:23.737739 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd023575-d8fe-4d5c-920a-83e528304bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:23.738243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd023575-d8fe-4d5c-920a-83e528304bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:23.738713 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd023575-d8fe-4d5c-920a-83e528304bb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:23.776546 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd023575-d8fe-4d5c-920a-83e528304bb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:23.779279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd023575-d8fe-4d5c-920a-83e528304bb6 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:23.798497 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81374f-fd53-4e65-a4f5-35f745e1081f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:23.840092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-fd023575-d8fe-4d5c-920a-83e528304bb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:23.840092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:23.840092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:23.840092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:23.840092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:23.840092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:23.840092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:23.840092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:23.840092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:23.840092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:23.840092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:23.840092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:23.840092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:23.840092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:23.840092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:23.840092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:23.841138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:23.841138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:23.841138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:23.841138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:23.841138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:23.841138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:23.841138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:23.841138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:23.841138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:23.841138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:23.841138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:23.841138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:23.841138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:23.841138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:23.841138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:23.841138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:23.841138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:23.842025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:23.842025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:23.842025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:23.842025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 374/742] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:23.849305 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd023575-d8fe-4d5c-920a-83e528304bb6 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:24.020082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caad890-ce44-4c08-851b-05a35d9f8ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:24.021691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caad890-ce44-4c08-851b-05a35d9f8ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:24.022156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caad890-ce44-4c08-851b-05a35d9f8ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:24.022633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caad890-ce44-4c08-851b-05a35d9f8ba6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:24.079162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:24.079770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd023575-d8fe-4d5c-920a-83e528304bb6 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:24.103694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:24.160446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caad890-ce44-4c08-851b-05a35d9f8ba6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:24.165178 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caad890-ce44-4c08-851b-05a35d9f8ba6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:24.170261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06654051-6c5b-46f1-a1cb-c4e17c3ba569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:24.172147 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06654051-6c5b-46f1-a1cb-c4e17c3ba569 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:24.172147 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06654051-6c5b-46f1-a1cb-c4e17c3ba569 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:24.172430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06654051-6c5b-46f1-a1cb-c4e17c3ba569 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:24.173747 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06654051-6c5b-46f1-a1cb-c4e17c3ba569 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:24.174418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06654051-6c5b-46f1-a1cb-c4e17c3ba569 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:24.177912 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06654051-6c5b-46f1-a1cb-c4e17c3ba569 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:24.203957 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 375/743] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:24.205699 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06654051-6c5b-46f1-a1cb-c4e17c3ba569 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:24.407703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 369/744] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:01:23 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 850 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 16:01:24.408958 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caad890-ce44-4c08-851b-05a35d9f8ba6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:24.462809 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06654051-6c5b-46f1-a1cb-c4e17c3ba569 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:24.534623 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:24.545936 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0e42a-15b5-4899-a7d7-912593e405ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:24.547548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0e42a-15b5-4899-a7d7-912593e405ce None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:24.548340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0e42a-15b5-4899-a7d7-912593e405ce None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:24.549197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0e42a-15b5-4899-a7d7-912593e405ce None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:24.551170 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0e42a-15b5-4899-a7d7-912593e405ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:24.555713 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0e42a-15b5-4899-a7d7-912593e405ce None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:24.613584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 376/745] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 6 16:01:24 2022] POST /identity/v3/services => generated 251 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:01:24.615459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0e42a-15b5-4899-a7d7-912593e405ce None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:24.635549 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caad890-ce44-4c08-851b-05a35d9f8ba6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:24.703249 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:24.741989 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7743dbe-6e80-484e-9a6b-479debc33f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:24.742643 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7743dbe-6e80-484e-9a6b-479debc33f6b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:24.743081 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7743dbe-6e80-484e-9a6b-479debc33f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:24.743356 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7743dbe-6e80-484e-9a6b-479debc33f6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:24.761225 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7743dbe-6e80-484e-9a6b-479debc33f6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:24.762760 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7743dbe-6e80-484e-9a6b-479debc33f6b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:24.787387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-e7743dbe-6e80-484e-9a6b-479debc33f6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:24.787387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:24.787387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:24.787387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:24.787387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:24.787387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:24.787387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:24.787387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:24.787387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:24.787387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:24.787387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:24.787387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:24.787387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:24.787387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:24.787387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:24.789128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:24.789128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:24.789128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:24.789128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:24.789128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:24.789128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:24.789128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:24.789128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:24.789128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:24.789128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:24.789128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:24.789128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:24.789128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:24.789128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:24.789128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:24.789128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:24.789128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:24.790963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:24.790963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:24.790963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:24.790963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:24.790963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:24.790963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:24.790963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:24.790963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:24.790963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 370/746] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:24.794970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7743dbe-6e80-484e-9a6b-479debc33f6b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:24.899399 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0e42a-15b5-4899-a7d7-912593e405ce None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:24.950370 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba9a4a-967b-48c5-8870-9f20b958a44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:24.951088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba9a4a-967b-48c5-8870-9f20b958a44c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:24.951371 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba9a4a-967b-48c5-8870-9f20b958a44c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:24.951640 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba9a4a-967b-48c5-8870-9f20b958a44c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:24.965482 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba9a4a-967b-48c5-8870-9f20b958a44c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:24.966918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba9a4a-967b-48c5-8870-9f20b958a44c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:24.983931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-c0ba9a4a-967b-48c5-8870-9f20b958a44c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:24.983931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:24.983931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:24.983931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:24.983931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:24.983931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:24.983931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:24.983931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:24.983931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:24.983931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:24.983931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:24.983931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:24.983931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:24.983931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:24.983931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:24.983931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:24.985825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:24.985825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:24.985825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:24.985825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:24.985825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:24.985825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:24.985825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:24.985825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:24.985825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:24.985825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:24.985825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:24.985825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:24.985825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:24.985825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:24.985825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:24.985825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:24.985825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:24.987644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:24.987644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:24.987644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:24.987644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 377/747] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:24.996188 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba9a4a-967b-48c5-8870-9f20b958a44c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:25.016507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7743dbe-6e80-484e-9a6b-479debc33f6b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:25.050803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:25.090490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451148e8-ffd4-47da-8198-5e1ac2bb7cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:25.092682 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451148e8-ffd4-47da-8198-5e1ac2bb7cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:25.093374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451148e8-ffd4-47da-8198-5e1ac2bb7cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:25.094075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451148e8-ffd4-47da-8198-5e1ac2bb7cc9 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:25.095705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451148e8-ffd4-47da-8198-5e1ac2bb7cc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:25.096606 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451148e8-ffd4-47da-8198-5e1ac2bb7cc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:25.100665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451148e8-ffd4-47da-8198-5e1ac2bb7cc9 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:25.125137 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 371/748] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:25.132291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451148e8-ffd4-47da-8198-5e1ac2bb7cc9 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:25.187779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:25.228209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba9a4a-967b-48c5-8870-9f20b958a44c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:25.294919 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3906983-0176-4bb6-ab20-4f52643443b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:25.296347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3906983-0176-4bb6-ab20-4f52643443b0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:25.297280 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3906983-0176-4bb6-ab20-4f52643443b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:25.298038 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3906983-0176-4bb6-ab20-4f52643443b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:25.299485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3906983-0176-4bb6-ab20-4f52643443b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:25.300258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3906983-0176-4bb6-ab20-4f52643443b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:25.304996 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3906983-0176-4bb6-ab20-4f52643443b0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:25.330926 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 378/749] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:25.334229 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ada92-5770-4854-87ea-cf2a78f382b1 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:25.335103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ada92-5770-4854-87ea-cf2a78f382b1 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:25.335835 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ada92-5770-4854-87ea-cf2a78f382b1 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:25.339464 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 379/750] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:25.347360 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ada92-5770-4854-87ea-cf2a78f382b1 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:25.359219 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451148e8-ffd4-47da-8198-5e1ac2bb7cc9 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:25.429353 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da588d4-f004-42e9-85ff-ecd3940c7825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:25.430824 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da588d4-f004-42e9-85ff-ecd3940c7825 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:25.431558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da588d4-f004-42e9-85ff-ecd3940c7825 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:25.432279 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da588d4-f004-42e9-85ff-ecd3940c7825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:25.433914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da588d4-f004-42e9-85ff-ecd3940c7825 None admin] RBAC: Adding query filter params (role.id=56d621cf28924933b6356795a6f8ba23, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=ba58a4914ddf418db9e8b9dce5784cf8) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:25.434812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da588d4-f004-42e9-85ff-ecd3940c7825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:25.437779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da588d4-f004-42e9-85ff-ecd3940c7825 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:25.451082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 372/751] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:01:25 2022] GET /identity/v3/role_assignments?user.id=ba58a4914ddf418db9e8b9dce5784cf8&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=56d621cf28924933b6356795a6f8ba23 => generated 259 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:25.454000 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ffded-7cee-45db-9cfb-a3a925206c7c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:25.454939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ffded-7cee-45db-9cfb-a3a925206c7c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:25.455736 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ffded-7cee-45db-9cfb-a3a925206c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:25.490113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-639ffded-7cee-45db-9cfb-a3a925206c7c None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:25.493074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-639ffded-7cee-45db-9cfb-a3a925206c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:25.586613 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ada92-5770-4854-87ea-cf2a78f382b1 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:25.595396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:25.674211 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7b09e-613b-4f6f-9d15-4d3b73fa6255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:25.676468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7b09e-613b-4f6f-9d15-4d3b73fa6255 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:25.678047 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7b09e-613b-4f6f-9d15-4d3b73fa6255 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:25.679433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7b09e-613b-4f6f-9d15-4d3b73fa6255 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:25.694934 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 373/752] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:25.702452 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7b09e-613b-4f6f-9d15-4d3b73fa6255 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:25.708204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7b09e-613b-4f6f-9d15-4d3b73fa6255 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:25.710674 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:25.732216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-3bd7b09e-613b-4f6f-9d15-4d3b73fa6255 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:25.732216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:25.732216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:25.732216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:25.732216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:25.732216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:25.732216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:25.732216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:25.732216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:25.732216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:25.732216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:25.732216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:25.732216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:25.732216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:25.732216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:25.733175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:25.733175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:25.733175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:25.733175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:25.733175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:25.733175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:25.733175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:25.733175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:25.733175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:25.733175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:25.733175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:25.733175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:25.733175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:25.733175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:25.733175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:25.733175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:25.733175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:25.734102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:25.734102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:25.734102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:25.734102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:25.734102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:25.734102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:25.734102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:25.734102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:25.734928 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 380/753] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:25.745813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ffded-7cee-45db-9cfb-a3a925206c7c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:25.958299 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ffded-7cee-45db-9cfb-a3a925206c7c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:26.043200 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e35870-ae8c-4585-8fff-2be02fa74d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:26.043970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e35870-ae8c-4585-8fff-2be02fa74d18 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:26.044473 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e35870-ae8c-4585-8fff-2be02fa74d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:26.044885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e35870-ae8c-4585-8fff-2be02fa74d18 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:26.045991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e35870-ae8c-4585-8fff-2be02fa74d18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:26.046423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e35870-ae8c-4585-8fff-2be02fa74d18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:26.048577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e35870-ae8c-4585-8fff-2be02fa74d18 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:26.064749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7b09e-613b-4f6f-9d15-4d3b73fa6255 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:26.075989 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 374/754] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:26.082172 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e35870-ae8c-4585-8fff-2be02fa74d18 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:26.125594 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:26.199054 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7b09e-613b-4f6f-9d15-4d3b73fa6255 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:26.237300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e35870-ae8c-4585-8fff-2be02fa74d18 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:26.272645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1241640-2e15-42ea-80a1-96297f78d841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:26.273956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1241640-2e15-42ea-80a1-96297f78d841 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:26.274460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1241640-2e15-42ea-80a1-96297f78d841 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:26.275112 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1241640-2e15-42ea-80a1-96297f78d841 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:26.283926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c54b1-5395-4b54-ba01-4e09f8870930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:26.284553 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c54b1-5395-4b54-ba01-4e09f8870930 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:26.284829 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c54b1-5395-4b54-ba01-4e09f8870930 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:26.285102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c54b1-5395-4b54-ba01-4e09f8870930 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:26.285870 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c54b1-5395-4b54-ba01-4e09f8870930 None admin] RBAC: Adding query filter params (role.id=ba05677650e046a48d5f19440d48fbf2, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=ece176d412574790a13d03dcca08f91a) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:26.286295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c54b1-5395-4b54-ba01-4e09f8870930 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:26.288411 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c54b1-5395-4b54-ba01-4e09f8870930 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:26.295450 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 375/755] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:01:26 2022] GET /identity/v3/role_assignments?user.id=ece176d412574790a13d03dcca08f91a&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:26.296696 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda798d-9f8a-4386-8a75-052a6c251c8a None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:26.296974 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda798d-9f8a-4386-8a75-052a6c251c8a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:26.297249 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda798d-9f8a-4386-8a75-052a6c251c8a None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:26.298991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 376/756] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:26.308590 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1241640-2e15-42ea-80a1-96297f78d841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:01:26.310173 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4595d76-9b71-4c31-84f8-b35d730a0ee5 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:26.310713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4595d76-9b71-4c31-84f8-b35d730a0ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:26.315479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4595d76-9b71-4c31-84f8-b35d730a0ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:26.316540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-c1241640-2e15-42ea-80a1-96297f78d841 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:01:26.316540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:26.316540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:26.316540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:26.316540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:26.316540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:26.316540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:26.316540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:26.316540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:26.316540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:26.316540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:26.316540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:26.316540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:01:26.316540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:01:26.316540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:01:26.316540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:01:26.317531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:01:26.317531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:01:26.317531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:01:26.317531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:01:26.317531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:26.317531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:26.317531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:26.317531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:26.317531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:26.317531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:26.317531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:26.317531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:26.317531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:26.317531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:26.317531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:26.317531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:26.319104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:26.319104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:26.319104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:26.319104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:26.319104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:01:26.319104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:01:26.319104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:01:26.319104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:01:26.319104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:01:26.319104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:01:26.319104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:01:26.319104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:26.319929 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 381/757] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:01:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:26.327856 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1241640-2e15-42ea-80a1-96297f78d841 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:26.381159 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c4595d76-9b71-4c31-84f8-b35d730a0ee5 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:26.383465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c4595d76-9b71-4c31-84f8-b35d730a0ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:26.415071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:26.529232 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1241640-2e15-42ea-80a1-96297f78d841 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:26.543684 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 377/758] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:01:26.590077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31148fb-1d54-4319-a269-67ecc6d13565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:26.591695 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31148fb-1d54-4319-a269-67ecc6d13565 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:26.592457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31148fb-1d54-4319-a269-67ecc6d13565 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:26.593227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31148fb-1d54-4319-a269-67ecc6d13565 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:26.594814 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31148fb-1d54-4319-a269-67ecc6d13565 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:26.595640 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31148fb-1d54-4319-a269-67ecc6d13565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:26.600125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31148fb-1d54-4319-a269-67ecc6d13565 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:26.620006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 382/759] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:01:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:26.628812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4595d76-9b71-4c31-84f8-b35d730a0ee5 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:26.679027 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:26.838865 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4595d76-9b71-4c31-84f8-b35d730a0ee5 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:26.890115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8af50d-0687-47ae-95ef-4894f1121895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:26.890692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8af50d-0687-47ae-95ef-4894f1121895 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:26.890981 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8af50d-0687-47ae-95ef-4894f1121895 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:26.891252 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8af50d-0687-47ae-95ef-4894f1121895 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:26.905997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8af50d-0687-47ae-95ef-4894f1121895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:26.907499 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8af50d-0687-47ae-95ef-4894f1121895 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:26.935462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-1d8af50d-0687-47ae-95ef-4894f1121895 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:26.935462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:26.935462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:26.935462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:26.935462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:26.935462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:26.935462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:26.935462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:26.935462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:26.935462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:26.935462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:26.935462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:26.935462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:26.935462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:26.935462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:26.935462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:26.936464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:26.936464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:26.936464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:26.936464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:26.936464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:26.936464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:26.936464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:26.936464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:26.936464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:26.936464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:26.936464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:26.936464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:26.936464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:26.936464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:26.936464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:26.936464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:26.936464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:26.937364 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:26.937364 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:26.937364 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:26.937364 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 378/760] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:26.947108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31148fb-1d54-4319-a269-67ecc6d13565 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:26.982223 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8af50d-0687-47ae-95ef-4894f1121895 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:27.142912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:27.172195 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31148fb-1d54-4319-a269-67ecc6d13565 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:27.196445 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8af50d-0687-47ae-95ef-4894f1121895 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:27.225097 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:27.253081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21a06e-21aa-4f9e-a468-0bc84f6efef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:27.254480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21a06e-21aa-4f9e-a468-0bc84f6efef1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:27.256234 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21a06e-21aa-4f9e-a468-0bc84f6efef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:27.256731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21a06e-21aa-4f9e-a468-0bc84f6efef1 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:27.257598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21a06e-21aa-4f9e-a468-0bc84f6efef1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:27.258605 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21a06e-21aa-4f9e-a468-0bc84f6efef1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:27.260741 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21a06e-21aa-4f9e-a468-0bc84f6efef1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:27.264225 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c9a2a-8370-4dd7-8347-d4053ea19d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:27.265186 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c9a2a-8370-4dd7-8347-d4053ea19d60 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:27.265742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c9a2a-8370-4dd7-8347-d4053ea19d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:27.266207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c9a2a-8370-4dd7-8347-d4053ea19d60 None admin] PATH_INFO: `/v3/services/network` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:27.268306 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c9a2a-8370-4dd7-8347-d4053ea19d60 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:27.269912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c9a2a-8370-4dd7-8347-d4053ea19d60 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:27.288269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-2d7c9a2a-8370-4dd7-8347-d4053ea19d60 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 16:01:27.288269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:27.288269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:01:27.288269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:01:27.288269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:01:27.288269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:01:27.288269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:01:27.288269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:27.288269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 16:01:27.288269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:27.288269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:01:27.288269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:27.288269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:27.288269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:27.288269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:27.288269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:27.288269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:27.289318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:27.289318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:27.289318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:27.289318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:27.289318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:27.289318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:27.289318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:01:27.289318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:01:27.289318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:01:27.289318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:01:27.289318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:27.289318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:27.289318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:27.289318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:27.289318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:27.289318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:27.290128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:27.290128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:27.290128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:27.290128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:27.290128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:27.290128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:27.290128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:27.290128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:27.290128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:27.290128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:27.290128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:01:27.290128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:27.290128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 16:01:27.290128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:27.295867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 383/761] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:27.302253 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 379/762] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:26 2022] GET /identity/v3/services/network => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:27.304033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21a06e-21aa-4f9e-a468-0bc84f6efef1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:27.314748 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c9a2a-8370-4dd7-8347-d4053ea19d60 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:27.443673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21a06e-21aa-4f9e-a468-0bc84f6efef1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:27.461856 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c9a2a-8370-4dd7-8347-d4053ea19d60 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:27.492700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459b717-2013-4f44-840d-9ad9d60fdf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:27.496461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459b717-2013-4f44-840d-9ad9d60fdf2d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:27.496461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459b717-2013-4f44-840d-9ad9d60fdf2d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:27.496461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459b717-2013-4f44-840d-9ad9d60fdf2d None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:27.497067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459b717-2013-4f44-840d-9ad9d60fdf2d None admin] RBAC: Adding query filter params (name=network) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:27.497067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459b717-2013-4f44-840d-9ad9d60fdf2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:27.498467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459b717-2013-4f44-840d-9ad9d60fdf2d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:27.515824 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 384/763] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 6 16:01:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:27.523145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459b717-2013-4f44-840d-9ad9d60fdf2d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:27.556890 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311085a-5910-461c-bda1-561081c5ebd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:27.557563 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311085a-5910-461c-bda1-561081c5ebd6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:27.557846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311085a-5910-461c-bda1-561081c5ebd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:27.558121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311085a-5910-461c-bda1-561081c5ebd6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:27.591575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311085a-5910-461c-bda1-561081c5ebd6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:27.593133 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311085a-5910-461c-bda1-561081c5ebd6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:27.608045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:27.628208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-9311085a-5910-461c-bda1-561081c5ebd6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:01:27.628208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:27.628208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:27.628208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:27.628208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:27.628208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:27.628208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:27.628208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:27.628208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:27.628208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:27.628208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:27.628208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:27.628208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:27.628208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:27.628208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:27.628208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:27.630413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:27.630413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:27.630413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:27.630413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:27.630413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:27.630413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:27.630413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:27.630413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:27.630413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:27.630413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:27.630413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:27.630413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:27.630413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:27.630413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:27.630413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:27.630413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:27.630413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:27.630413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:27.633167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:27.633167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:27.633167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:27.633167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:27.633167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:27.633167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:27.633167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:27.633167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:27.633167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:01:27.633167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:27.635798 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 380/764] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Sep 6 16:01:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:27.638970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311085a-5910-461c-bda1-561081c5ebd6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:27.654553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:27.778046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459b717-2013-4f44-840d-9ad9d60fdf2d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:27.841882 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d179d47-e4f0-4d5d-b04a-9b4c02068594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:27.842989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d179d47-e4f0-4d5d-b04a-9b4c02068594 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:27.843527 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d179d47-e4f0-4d5d-b04a-9b4c02068594 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:27.844056 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d179d47-e4f0-4d5d-b04a-9b4c02068594 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:27.845178 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d179d47-e4f0-4d5d-b04a-9b4c02068594 None admin] RBAC: Adding query filter params (type=network) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:27.845791 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d179d47-e4f0-4d5d-b04a-9b4c02068594 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:27.848388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d179d47-e4f0-4d5d-b04a-9b4c02068594 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:27.865501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 385/765] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 6 16:01:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:27.865501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1dea8-fa72-41d1-b418-4f91b46707e4 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:27.865501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1dea8-fa72-41d1-b418-4f91b46707e4 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:27.865501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1dea8-fa72-41d1-b418-4f91b46707e4 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:27.865501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 386/766] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:27.867530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1dea8-fa72-41d1-b418-4f91b46707e4 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:27.880614 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311085a-5910-461c-bda1-561081c5ebd6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:27.952271 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670af3c6-5315-4b33-95db-f0a1231cabf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:27.953536 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670af3c6-5315-4b33-95db-f0a1231cabf2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:27.954807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670af3c6-5315-4b33-95db-f0a1231cabf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:27.955480 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670af3c6-5315-4b33-95db-f0a1231cabf2 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:27.957306 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670af3c6-5315-4b33-95db-f0a1231cabf2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:27.958616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670af3c6-5315-4b33-95db-f0a1231cabf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:27.963067 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670af3c6-5315-4b33-95db-f0a1231cabf2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:28.005400 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 381/767] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Sep 6 16:01:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:28.008599 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07338c03-aeda-46b5-8884-2d982129427f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:28.008599 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07338c03-aeda-46b5-8884-2d982129427f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:28.008996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07338c03-aeda-46b5-8884-2d982129427f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:28.043576 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-07338c03-aeda-46b5-8884-2d982129427f None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:28.046862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-07338c03-aeda-46b5-8884-2d982129427f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:28.072845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1dea8-fa72-41d1-b418-4f91b46707e4 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:28.098598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:28.148253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb9b4b-2d87-4fe2-8a1b-58012eefae48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:28.149209 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:28.149684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb9b4b-2d87-4fe2-8a1b-58012eefae48 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:28.149684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb9b4b-2d87-4fe2-8a1b-58012eefae48 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:28.150097 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb9b4b-2d87-4fe2-8a1b-58012eefae48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:28.151238 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb9b4b-2d87-4fe2-8a1b-58012eefae48 None admin] RBAC: Adding query filter params (interface=public, service_id=556720dae6554d28976085e39532e775, region_id=RegionOne) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:28.151590 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb9b4b-2d87-4fe2-8a1b-58012eefae48 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:28.153822 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb9b4b-2d87-4fe2-8a1b-58012eefae48 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:28.167401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 387/768] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Sep 6 16:01:27 2022] GET /identity/v3/endpoints?service_id=556720dae6554d28976085e39532e775&interface=public®ion_id=RegionOne => generated 194 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:28.175093 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 382/769] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:28.176640 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb9b4b-2d87-4fe2-8a1b-58012eefae48 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:28.177946 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322122d5-5105-407b-a765-e4ccd5e76472 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:28.178397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322122d5-5105-407b-a765-e4ccd5e76472 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:28.182756 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322122d5-5105-407b-a765-e4ccd5e76472 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:28.182756 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 383/770] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:28.183049 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322122d5-5105-407b-a765-e4ccd5e76472 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:28.343405 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322122d5-5105-407b-a765-e4ccd5e76472 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:28.346063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb9b4b-2d87-4fe2-8a1b-58012eefae48 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:28.388261 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6353a82-2da1-47fa-b1b3-63bd9a56f800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:28.389038 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6353a82-2da1-47fa-b1b3-63bd9a56f800 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:28.389335 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6353a82-2da1-47fa-b1b3-63bd9a56f800 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:28.389611 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6353a82-2da1-47fa-b1b3-63bd9a56f800 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:28.390618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6353a82-2da1-47fa-b1b3-63bd9a56f800 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:28.392354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6353a82-2da1-47fa-b1b3-63bd9a56f800 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:28.399863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 384/771] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Sep 6 16:01:28 2022] GET /identity/v3/services => generated 772 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:28.401830 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527b089-d15c-486d-b823-ab3d39c5d58a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:28.402291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527b089-d15c-486d-b823-ab3d39c5d58a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:28.402735 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527b089-d15c-486d-b823-ab3d39c5d58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:28.408305 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd757e2-194f-4194-a464-4571586f2ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:28.409366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd757e2-194f-4194-a464-4571586f2ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:28.409867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd757e2-194f-4194-a464-4571586f2ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:28.410541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd757e2-194f-4194-a464-4571586f2ef9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:28.425368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a527b089-d15c-486d-b823-ab3d39c5d58a None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:28.427265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a527b089-d15c-486d-b823-ab3d39c5d58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:28.432013 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd757e2-194f-4194-a464-4571586f2ef9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:28.434684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd757e2-194f-4194-a464-4571586f2ef9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:28.462292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-8dd757e2-194f-4194-a464-4571586f2ef9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:28.462292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:28.462292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:28.462292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:28.462292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:28.462292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:28.462292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:28.462292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:28.462292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:28.462292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:28.462292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:28.462292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:28.462292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:28.462292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:28.462292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:28.462292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:28.463363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:28.463363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:28.463363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:28.463363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:28.463363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:28.463363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:28.463363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:28.463363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:28.463363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:28.463363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:28.463363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:28.463363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:28.463363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:28.463363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:28.463363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:28.463363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:28.463363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:28.464229 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:28.464229 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:28.464229 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:28.464836 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 388/772] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:28.471529 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd757e2-194f-4194-a464-4571586f2ef9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:28.505186 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:28.510314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:28.537256 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 385/773] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:28.539556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527b089-d15c-486d-b823-ab3d39c5d58a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:28.664971 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd757e2-194f-4194-a464-4571586f2ef9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:28.725077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a8bd8-b713-4f8b-843f-ebe9e984c962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:28.725827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a8bd8-b713-4f8b-843f-ebe9e984c962 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:28.726402 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a8bd8-b713-4f8b-843f-ebe9e984c962 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:28.726774 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a8bd8-b713-4f8b-843f-ebe9e984c962 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:28.727380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a8bd8-b713-4f8b-843f-ebe9e984c962 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:28.728801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a8bd8-b713-4f8b-843f-ebe9e984c962 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:28.741534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-237a8bd8-b713-4f8b-843f-ebe9e984c962 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 16:01:28.741534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:28.741534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:01:28.741534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:01:28.741534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:01:28.741534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:01:28.741534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:01:28.741534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:28.741534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 16:01:28.741534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:28.741534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:01:28.741534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:28.741534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:28.741534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:28.741534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:28.741534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:28.741534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:28.742564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:28.742564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:28.742564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:28.742564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:28.742564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:28.742564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:28.742564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:01:28.742564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:01:28.742564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:01:28.742564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:01:28.742564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:28.742564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:28.742564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:28.742564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:28.742564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:28.742564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:28.743407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:28.743407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:28.743407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:28.743407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:28.743407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:28.743407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:28.743407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:28.743407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:28.743407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:28.743407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:28.743407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:01:28.743407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:28.743407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 16:01:28.743407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:28.743407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 389/774] 10.222.0.26 () {56 vars in 1129 bytes} [Tue Sep 6 16:01:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:28.752048 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a8bd8-b713-4f8b-843f-ebe9e984c962 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:28.774893 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527b089-d15c-486d-b823-ab3d39c5d58a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:28.844900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a808b-676e-455c-ada8-acf694e14b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:28.845880 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a808b-676e-455c-ada8-acf694e14b5d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:28.846364 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a808b-676e-455c-ada8-acf694e14b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:28.846890 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a808b-676e-455c-ada8-acf694e14b5d None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:28.848083 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a808b-676e-455c-ada8-acf694e14b5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:28.848697 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a808b-676e-455c-ada8-acf694e14b5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:28.851337 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a808b-676e-455c-ada8-acf694e14b5d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:28.874651 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 386/775] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:28.876095 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a808b-676e-455c-ada8-acf694e14b5d None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:28.907298 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:28.930727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a8bd8-b713-4f8b-843f-ebe9e984c962 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:28.935459 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:29.002079 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37b421-14f1-4ebd-b587-88742bf3671e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:29.003626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37b421-14f1-4ebd-b587-88742bf3671e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:29.004216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37b421-14f1-4ebd-b587-88742bf3671e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:29.005641 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37b421-14f1-4ebd-b587-88742bf3671e None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:29.007222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37b421-14f1-4ebd-b587-88742bf3671e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:29.008032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37b421-14f1-4ebd-b587-88742bf3671e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:29.011142 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37b421-14f1-4ebd-b587-88742bf3671e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:29.023646 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 390/776] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Sep 6 16:01:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:29.025473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37b421-14f1-4ebd-b587-88742bf3671e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:29.058127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a808b-676e-455c-ada8-acf694e14b5d None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:29.132585 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252fcf6-1f45-4e8c-9d0d-3f5dd5755e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:29.133967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252fcf6-1f45-4e8c-9d0d-3f5dd5755e33 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:29.134467 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252fcf6-1f45-4e8c-9d0d-3f5dd5755e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:29.135475 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252fcf6-1f45-4e8c-9d0d-3f5dd5755e33 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:29.150809 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252fcf6-1f45-4e8c-9d0d-3f5dd5755e33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:01:29.162571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-2252fcf6-1f45-4e8c-9d0d-3f5dd5755e33 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:01:29.162571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:29.162571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:29.162571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:29.162571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:29.162571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:29.162571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:29.162571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:29.162571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:29.162571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:29.162571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:29.162571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:29.162571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:01:29.162571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:01:29.162571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:01:29.162571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:01:29.163971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:01:29.163971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:01:29.163971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:01:29.163971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:01:29.163971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:29.163971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:29.163971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:29.163971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:29.163971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:29.163971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:29.163971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:29.163971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:29.163971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:29.163971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:29.163971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:29.163971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:29.165242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:29.165242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:29.165242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:29.165242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:29.165242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:01:29.165242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:01:29.165242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:01:29.165242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:01:29.165242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:01:29.165242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:01:29.165242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:01:29.165242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:29.165242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 387/777] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:01:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:29.173353 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252fcf6-1f45-4e8c-9d0d-3f5dd5755e33 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:29.237007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37b421-14f1-4ebd-b587-88742bf3671e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:29.279513 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739af05-cfc7-49a0-8b0b-00f5c5cf7713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:29.280167 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739af05-cfc7-49a0-8b0b-00f5c5cf7713 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:29.280445 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739af05-cfc7-49a0-8b0b-00f5c5cf7713 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:29.280718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739af05-cfc7-49a0-8b0b-00f5c5cf7713 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:29.289542 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739af05-cfc7-49a0-8b0b-00f5c5cf7713 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:29.291271 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739af05-cfc7-49a0-8b0b-00f5c5cf7713 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:29.305341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-8739af05-cfc7-49a0-8b0b-00f5c5cf7713 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:29.305341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:29.305341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:29.305341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:29.305341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:29.305341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:29.305341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:29.305341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:29.305341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:29.305341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:29.305341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:29.305341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:29.305341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:29.305341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:29.305341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:29.306389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:29.306389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:29.306389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:29.306389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:29.306389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:29.306389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:29.306389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:29.306389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:29.306389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:29.306389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:29.306389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:29.306389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:29.306389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:29.306389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:29.306389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:29.306389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:29.306389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:29.307327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:29.307327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:29.307327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:29.307327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:29.307327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:29.307327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:29.307327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:29.307327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:29.307327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 391/778] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:29.307819 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739af05-cfc7-49a0-8b0b-00f5c5cf7713 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:29.321884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252fcf6-1f45-4e8c-9d0d-3f5dd5755e33 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:29.324630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:29.331776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:29.387124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d05534-353b-4b21-b8a7-7589cd5dd68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:29.388092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d05534-353b-4b21-b8a7-7589cd5dd68b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:29.388590 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d05534-353b-4b21-b8a7-7589cd5dd68b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:29.389154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d05534-353b-4b21-b8a7-7589cd5dd68b None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:29.390511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d05534-353b-4b21-b8a7-7589cd5dd68b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:29.391092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d05534-353b-4b21-b8a7-7589cd5dd68b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:29.393708 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d05534-353b-4b21-b8a7-7589cd5dd68b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:29.413087 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 388/779] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Sep 6 16:01:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:29.414842 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d05534-353b-4b21-b8a7-7589cd5dd68b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:29.451866 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739af05-cfc7-49a0-8b0b-00f5c5cf7713 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:29.502150 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27b3f7-7e63-4fd6-8810-ab68b3df514f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:29.503235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27b3f7-7e63-4fd6-8810-ab68b3df514f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:29.503746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27b3f7-7e63-4fd6-8810-ab68b3df514f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:29.505134 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27b3f7-7e63-4fd6-8810-ab68b3df514f None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:29.506592 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27b3f7-7e63-4fd6-8810-ab68b3df514f None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:29.507169 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27b3f7-7e63-4fd6-8810-ab68b3df514f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:29.508805 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27b3f7-7e63-4fd6-8810-ab68b3df514f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:29.518775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 392/780] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:01:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:29.522815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27b3f7-7e63-4fd6-8810-ab68b3df514f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:29.587763 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d05534-353b-4b21-b8a7-7589cd5dd68b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:29.629171 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:29.656963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27b3f7-7e63-4fd6-8810-ab68b3df514f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:29.657615 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1cb1ef-d0ea-4330-a2e5-5447590ccd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:29.658639 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1cb1ef-d0ea-4330-a2e5-5447590ccd69 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:29.659204 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1cb1ef-d0ea-4330-a2e5-5447590ccd69 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:29.659688 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1cb1ef-d0ea-4330-a2e5-5447590ccd69 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:29.660919 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1cb1ef-d0ea-4330-a2e5-5447590ccd69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:29.661993 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1cb1ef-d0ea-4330-a2e5-5447590ccd69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:29.664797 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1cb1ef-d0ea-4330-a2e5-5447590ccd69 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:29.693601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 389/781] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:29.695904 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9754e-4d2a-4122-b246-5c96df6269ea None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:29.696412 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9754e-4d2a-4122-b246-5c96df6269ea None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:29.696909 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9754e-4d2a-4122-b246-5c96df6269ea None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:29.702645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 390/782] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:29.702645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9754e-4d2a-4122-b246-5c96df6269ea None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:29.719066 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f4072-09b7-43ef-b7e9-82fe0c8c173c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:29.719694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f4072-09b7-43ef-b7e9-82fe0c8c173c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:29.719971 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f4072-09b7-43ef-b7e9-82fe0c8c173c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:29.720244 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f4072-09b7-43ef-b7e9-82fe0c8c173c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:29.720957 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f4072-09b7-43ef-b7e9-82fe0c8c173c None admin] RBAC: Adding query filter params (interface=public, service_id=9abc536e3fea405380678fd9f38a38a9, region_id=RegionOne) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:29.721268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f4072-09b7-43ef-b7e9-82fe0c8c173c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:29.722725 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f4072-09b7-43ef-b7e9-82fe0c8c173c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:29.730800 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 393/783] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Sep 6 16:01:29 2022] GET /identity/v3/endpoints?service_id=9abc536e3fea405380678fd9f38a38a9&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:29.731718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f4072-09b7-43ef-b7e9-82fe0c8c173c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:29.751228 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:29.876043 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9754e-4d2a-4122-b246-5c96df6269ea None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:29.896501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f4072-09b7-43ef-b7e9-82fe0c8c173c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:29.924409 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efb0b6-4686-4fed-8ec8-a1bef3dcfdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:29.926105 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efb0b6-4686-4fed-8ec8-a1bef3dcfdd9 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:29.927649 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efb0b6-4686-4fed-8ec8-a1bef3dcfdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:29.928447 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efb0b6-4686-4fed-8ec8-a1bef3dcfdd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:29.947907 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efb0b6-4686-4fed-8ec8-a1bef3dcfdd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:29.950692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efb0b6-4686-4fed-8ec8-a1bef3dcfdd9 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:29.954102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b6977-5439-4dd6-a4a5-7a28f042e7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:29.956004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b6977-5439-4dd6-a4a5-7a28f042e7ad None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:29.956004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b6977-5439-4dd6-a4a5-7a28f042e7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:29.956004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b6977-5439-4dd6-a4a5-7a28f042e7ad None admin] PATH_INFO: `/v3/projects/0138d62d5879495499a17e20310d4f61/users/6afd4f6ffdae4b16aa5ba757e68c18c4/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:29.957521 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b6977-5439-4dd6-a4a5-7a28f042e7ad None admin] RBAC: Authorizing `identity:create_grant(project_id=0138d62d5879495499a17e20310d4f61, user_id=6afd4f6ffdae4b16aa5ba757e68c18c4, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:29.959899 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b6977-5439-4dd6-a4a5-7a28f042e7ad None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:29.971008 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:29.986250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-b1efb0b6-4686-4fed-8ec8-a1bef3dcfdd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:29.986250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:29.986250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:29.986250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:29.986250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:29.986250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:29.986250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:29.986250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:29.986250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:29.986250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:29.986250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:29.986250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:29.986250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:29.986250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:29.986250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:29.986250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:29.988127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:29.988127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:29.988127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:29.988127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:29.988127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:29.988127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:29.988127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:29.988127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:29.988127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:29.988127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:29.988127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:29.988127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:29.988127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:29.988127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:29.988127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:29.988127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:29.988127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:29.990334 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:29.990334 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:29.990334 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:29.990770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 394/784] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:01:29 2022] PUT /identity/v3/projects/0138d62d5879495499a17e20310d4f61/users/6afd4f6ffdae4b16aa5ba757e68c18c4/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:01:29.997302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2372ae6-419a-4f00-abaa-681662a43047 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:29.997302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2372ae6-419a-4f00-abaa-681662a43047 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:29.997302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2372ae6-419a-4f00-abaa-681662a43047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:29.997630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 391/785] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:29.997630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efb0b6-4686-4fed-8ec8-a1bef3dcfdd9 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:30.027545 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-a2372ae6-419a-4f00-abaa-681662a43047 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:30.030467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a2372ae6-419a-4f00-abaa-681662a43047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:30.117001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:30.123687 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 395/786] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:30.125876 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2372ae6-419a-4f00-abaa-681662a43047 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:30.152339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efb0b6-4686-4fed-8ec8-a1bef3dcfdd9 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:30.194425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46372956-dbc6-4baa-9137-c8d6e3b4d7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:30.195274 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46372956-dbc6-4baa-9137-c8d6e3b4d7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:30.195715 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46372956-dbc6-4baa-9137-c8d6e3b4d7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:30.196023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46372956-dbc6-4baa-9137-c8d6e3b4d7a7 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:30.196670 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46372956-dbc6-4baa-9137-c8d6e3b4d7a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:30.198218 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46372956-dbc6-4baa-9137-c8d6e3b4d7a7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:30.207035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 392/787] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Sep 6 16:01:29 2022] GET /identity/v3/services => generated 772 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:30.313004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2372ae6-419a-4f00-abaa-681662a43047 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:30.355738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:30.367837 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf47b3-1e3e-4dbc-92f4-0d9180ee1a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:30.369705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf47b3-1e3e-4dbc-92f4-0d9180ee1a5c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:30.370551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf47b3-1e3e-4dbc-92f4-0d9180ee1a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:30.371379 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf47b3-1e3e-4dbc-92f4-0d9180ee1a5c None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:30.372950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf47b3-1e3e-4dbc-92f4-0d9180ee1a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:30.373889 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf47b3-1e3e-4dbc-92f4-0d9180ee1a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:30.377088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf47b3-1e3e-4dbc-92f4-0d9180ee1a5c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:30.400517 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 396/788] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:30.410833 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46372956-dbc6-4baa-9137-c8d6e3b4d7a7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:30.567749 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46372956-dbc6-4baa-9137-c8d6e3b4d7a7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:30.583778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf47b3-1e3e-4dbc-92f4-0d9180ee1a5c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:30.619094 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830d499-14e0-4648-ad55-48f18e9981f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:30.619852 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830d499-14e0-4648-ad55-48f18e9981f1 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:30.620137 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830d499-14e0-4648-ad55-48f18e9981f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:30.620407 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830d499-14e0-4648-ad55-48f18e9981f1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:30.630811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830d499-14e0-4648-ad55-48f18e9981f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:30.632665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830d499-14e0-4648-ad55-48f18e9981f1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:30.646806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-a830d499-14e0-4648-ad55-48f18e9981f1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:01:30.646806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:30.646806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:30.646806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:30.646806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:30.646806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:30.646806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:30.646806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:30.646806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:30.646806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:30.646806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:30.646806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:30.646806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:30.646806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:30.646806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:30.646806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:30.648600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:30.648600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:30.648600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:30.648600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:30.648600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:30.648600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:30.648600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:30.648600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:30.648600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:30.648600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:30.648600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:30.648600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:30.648600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:30.648600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:30.648600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:30.648600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:30.648600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:30.648600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:30.650287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:30.650287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:30.650287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:30.650287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:30.650287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:30.650287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:30.650287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:30.650287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:30.650287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:01:30.650287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:30.650287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 393/789] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 6 16:01:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:30.656757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830d499-14e0-4648-ad55-48f18e9981f1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:30.664982 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:30.747085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf47b3-1e3e-4dbc-92f4-0d9180ee1a5c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:30.828977 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0c46a-4d40-4e1b-8692-184ac98d0161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:30.830047 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0c46a-4d40-4e1b-8692-184ac98d0161 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:30.830654 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0c46a-4d40-4e1b-8692-184ac98d0161 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:30.831172 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0c46a-4d40-4e1b-8692-184ac98d0161 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:30.841662 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830d499-14e0-4648-ad55-48f18e9981f1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:30.876333 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0c46a-4d40-4e1b-8692-184ac98d0161 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:01:30.882763 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b066c-698c-4f8b-a106-749269b416ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:30.883384 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b066c-698c-4f8b-a106-749269b416ff None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:30.890695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b066c-698c-4f8b-a106-749269b416ff None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:30.890695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b066c-698c-4f8b-a106-749269b416ff None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:30.890695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b066c-698c-4f8b-a106-749269b416ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:30.890695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b066c-698c-4f8b-a106-749269b416ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:30.890695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b066c-698c-4f8b-a106-749269b416ff None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:30.898052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-16c0c46a-4d40-4e1b-8692-184ac98d0161 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:01:30.898052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:30.898052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:30.898052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:30.898052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:30.898052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:30.898052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:30.898052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:30.898052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:30.898052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:30.898052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:30.898052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:30.898052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:01:30.898052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:01:30.898052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:01:30.898052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:01:30.899945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:01:30.899945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:01:30.899945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:01:30.899945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:01:30.899945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:30.899945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:30.899945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:30.899945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:30.899945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:30.899945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:30.899945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:30.899945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:30.899945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:30.899945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:30.899945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:30.899945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:30.901616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:30.901616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:30.901616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:30.901616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:30.901616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:01:30.901616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:01:30.901616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:01:30.901616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:01:30.901616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:01:30.901616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:01:30.901616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:01:30.901616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:30.908441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 394/790] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Sep 6 16:01:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:30.911942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 397/791] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:01:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:30.914738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b066c-698c-4f8b-a106-749269b416ff None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:30.920077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0c46a-4d40-4e1b-8692-184ac98d0161 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:31.004170 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:31.055483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:31.100000 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0c46a-4d40-4e1b-8692-184ac98d0161 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:31.133089 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b066c-698c-4f8b-a106-749269b416ff None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:31.158884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3746ccf-d697-4c1c-ab73-e36759bd54d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:31.159851 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3746ccf-d697-4c1c-ab73-e36759bd54d4 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:31.160336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3746ccf-d697-4c1c-ab73-e36759bd54d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:31.160789 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3746ccf-d697-4c1c-ab73-e36759bd54d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:31.162451 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3746ccf-d697-4c1c-ab73-e36759bd54d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:31.162996 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3746ccf-d697-4c1c-ab73-e36759bd54d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:31.165157 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3746ccf-d697-4c1c-ab73-e36759bd54d4 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:31.187192 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 398/792] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:01:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:31.189595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203ebb4-4084-4244-b1c1-78c1663439dd None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:31.190087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203ebb4-4084-4244-b1c1-78c1663439dd None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:31.190570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203ebb4-4084-4244-b1c1-78c1663439dd None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:31.195211 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234921a-2023-45a3-bf9e-b5daf8ba1659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:31.195961 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234921a-2023-45a3-bf9e-b5daf8ba1659 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:31.196339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234921a-2023-45a3-bf9e-b5daf8ba1659 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:31.196530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 399/793] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:31.196794 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234921a-2023-45a3-bf9e-b5daf8ba1659 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:31.197820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203ebb4-4084-4244-b1c1-78c1663439dd None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:31.209527 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234921a-2023-45a3-bf9e-b5daf8ba1659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:31.211391 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234921a-2023-45a3-bf9e-b5daf8ba1659 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:31.229082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-d234921a-2023-45a3-bf9e-b5daf8ba1659 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:31.229082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:31.229082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:31.229082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:31.229082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:31.229082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:31.229082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:31.229082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:31.229082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:31.229082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:31.229082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:31.229082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:31.229082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:31.229082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:31.229082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:31.229082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:31.231264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:31.231264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:31.231264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:31.231264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:31.231264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:31.231264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:31.231264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:31.231264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:31.231264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:31.231264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:31.231264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:31.231264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:31.231264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:31.231264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:31.231264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:31.231264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:31.231264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:31.234512 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:31.234512 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:31.234512 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:31.234512 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 395/794] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:31.239636 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc255a-77d1-455d-8c9b-9c8638dfa664 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:31.239636 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc255a-77d1-455d-8c9b-9c8638dfa664 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:31.239636 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc255a-77d1-455d-8c9b-9c8638dfa664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:31.259576 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f8dc255a-77d1-455d-8c9b-9c8638dfa664 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:31.261115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f8dc255a-77d1-455d-8c9b-9c8638dfa664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:31.345654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 396/795] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:31.346534 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc255a-77d1-455d-8c9b-9c8638dfa664 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:31.396709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203ebb4-4084-4244-b1c1-78c1663439dd None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:31.399321 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:31.426006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:31.479844 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b83476-7434-4b38-9ae9-b0ed0054239c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:31.480315 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc255a-77d1-455d-8c9b-9c8638dfa664 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:31.480894 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b83476-7434-4b38-9ae9-b0ed0054239c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:31.481380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b83476-7434-4b38-9ae9-b0ed0054239c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:31.481867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b83476-7434-4b38-9ae9-b0ed0054239c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:31.503558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b83476-7434-4b38-9ae9-b0ed0054239c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:31.506763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b83476-7434-4b38-9ae9-b0ed0054239c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:31.526478 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5db523-de4e-472a-b4af-f38cbd0fbc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:31.527167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5db523-de4e-472a-b4af-f38cbd0fbc96 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:31.527438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5db523-de4e-472a-b4af-f38cbd0fbc96 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:31.527727 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5db523-de4e-472a-b4af-f38cbd0fbc96 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:31.528387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5db523-de4e-472a-b4af-f38cbd0fbc96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:31.528693 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5db523-de4e-472a-b4af-f38cbd0fbc96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:31.530120 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5db523-de4e-472a-b4af-f38cbd0fbc96 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:31.536485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-08b83476-7434-4b38-9ae9-b0ed0054239c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:31.536485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:31.536485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:31.536485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:31.536485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:31.536485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:31.536485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:31.536485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:31.536485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:31.536485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:31.536485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:31.536485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:31.536485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:31.536485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:31.536485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:31.536485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:31.538347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:31.538347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:31.538347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:31.538347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:31.538347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:31.538347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:31.538347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:31.538347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:31.538347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:31.538347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:31.538347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:31.538347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:31.538347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:31.538347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:31.538347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:31.538347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:31.538347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:31.540132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:31.540132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:31.540132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:31.540132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 400/796] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:31.545135 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 397/797] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:31.547935 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b83476-7434-4b38-9ae9-b0ed0054239c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:31.551050 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5db523-de4e-472a-b4af-f38cbd0fbc96 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:31.687106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5db523-de4e-472a-b4af-f38cbd0fbc96 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:31.690768 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:31.748979 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3cdba8-8f7d-4e76-9301-56b085bd5747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:31.749635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cdba8-8f7d-4e76-9301-56b085bd5747 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:31.749941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cdba8-8f7d-4e76-9301-56b085bd5747 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:31.750329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cdba8-8f7d-4e76-9301-56b085bd5747 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:31.758681 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3cdba8-8f7d-4e76-9301-56b085bd5747 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:31.760172 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3cdba8-8f7d-4e76-9301-56b085bd5747 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:31.774336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-fd3cdba8-8f7d-4e76-9301-56b085bd5747 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:31.774336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:31.774336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:31.774336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:31.774336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:31.774336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:31.774336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:31.774336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:31.774336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:31.774336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:31.774336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:31.774336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:31.774336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:31.774336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:31.774336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:31.776164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:31.776164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:31.776164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:31.776164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:31.776164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:31.776164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:31.776164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:31.776164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:31.776164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:31.776164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:31.776164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:31.776164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:31.776164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:31.776164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:31.776164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:31.776164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:31.776164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:31.777897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:31.777897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:31.777897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:31.777897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:31.777897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:31.777897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:31.777897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:31.777897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:31.777897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 398/798] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:31.777897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3cdba8-8f7d-4e76-9301-56b085bd5747 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:31.816363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b83476-7434-4b38-9ae9-b0ed0054239c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:31.885947 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1209e-ac14-4323-8744-816629234084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:31.887181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1209e-ac14-4323-8744-816629234084 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:31.887670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1209e-ac14-4323-8744-816629234084 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:31.888168 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1209e-ac14-4323-8744-816629234084 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:31.889435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1209e-ac14-4323-8744-816629234084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:31.889985 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1209e-ac14-4323-8744-816629234084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:31.892880 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1209e-ac14-4323-8744-816629234084 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:31.917740 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3cdba8-8f7d-4e76-9301-56b085bd5747 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:31.923016 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 401/799] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:31.924651 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1209e-ac14-4323-8744-816629234084 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:31.958772 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52e7e1-9287-417f-8c01-b450658a1b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:31.959536 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52e7e1-9287-417f-8c01-b450658a1b5e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:31.959811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52e7e1-9287-417f-8c01-b450658a1b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:31.960091 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52e7e1-9287-417f-8c01-b450658a1b5e None admin] PATH_INFO: `/v3/services/network` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:31.960695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52e7e1-9287-417f-8c01-b450658a1b5e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:31.962217 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52e7e1-9287-417f-8c01-b450658a1b5e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:31.972154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-ce52e7e1-9287-417f-8c01-b450658a1b5e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 16:01:31.972154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:31.972154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:01:31.972154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:01:31.972154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:01:31.972154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:01:31.972154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:01:31.972154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:31.972154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 16:01:31.972154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:31.972154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:01:31.972154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:31.972154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:31.972154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:31.972154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:31.972154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:31.972154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:31.975247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:31.975247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:31.975247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:31.975247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:31.975247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:31.975247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:31.975247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:01:31.975247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:01:31.975247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:01:31.975247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:01:31.975247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:31.975247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:31.975247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:31.975247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:31.975247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:31.975247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:31.978723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:31.978723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:31.978723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:31.978723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:31.978723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:31.978723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:31.978723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:31.978723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:31.978723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:31.978723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:31.978723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:01:31.978723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:31.978723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 16:01:31.978723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:31.981073 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 399/800] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:31 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:31.981073 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52e7e1-9287-417f-8c01-b450658a1b5e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:31.981748 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:32.019179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:32.096685 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52e7e1-9287-417f-8c01-b450658a1b5e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:32.118506 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1209e-ac14-4323-8744-816629234084 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:32.139712 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890b6e5-162d-4709-80e9-1155f41b50cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:32.140363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890b6e5-162d-4709-80e9-1155f41b50cf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:32.140643 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890b6e5-162d-4709-80e9-1155f41b50cf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:32.140914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890b6e5-162d-4709-80e9-1155f41b50cf None admin] PATH_INFO: `/v3/users/swift` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:32.150102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890b6e5-162d-4709-80e9-1155f41b50cf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:32.151726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890b6e5-162d-4709-80e9-1155f41b50cf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:32.166126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-2890b6e5-162d-4709-80e9-1155f41b50cf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:01:32.166126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:32.166126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:32.166126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:32.166126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:32.166126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:32.166126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:32.166126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:32.166126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:32.166126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:32.166126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:32.166126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:32.166126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:32.166126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:32.166126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:32.166126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:32.168016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:32.168016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:32.168016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:32.168016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:32.168016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:32.168016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:32.168016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:32.168016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:32.168016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:32.168016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:32.168016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:32.168016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:32.168016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:32.168016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:32.168016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:32.168016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:32.168016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:32.168016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:32.169899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:32.169899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:32.169899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:32.169899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:32.169899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:32.169899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:32.169899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:32.169899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:32.169899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:01:32.169899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:32.169899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 400/801] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:01:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:32.169899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890b6e5-162d-4709-80e9-1155f41b50cf None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:32.190642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a5faf-5b93-4342-b429-c118358e3725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:32.191853 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a5faf-5b93-4342-b429-c118358e3725 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:32.192943 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a5faf-5b93-4342-b429-c118358e3725 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:32.193511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a5faf-5b93-4342-b429-c118358e3725 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:32.194930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a5faf-5b93-4342-b429-c118358e3725 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:32.195720 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a5faf-5b93-4342-b429-c118358e3725 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:32.198571 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a5faf-5b93-4342-b429-c118358e3725 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:32.230973 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 402/802] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:32.233038 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a5faf-5b93-4342-b429-c118358e3725 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:32.303980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:32.336230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890b6e5-162d-4709-80e9-1155f41b50cf None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:32.336665 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:32.354276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a5faf-5b93-4342-b429-c118358e3725 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:32.392399 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efadd5-5c6e-4666-a9cf-26077d97821c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:32.393193 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efadd5-5c6e-4666-a9cf-26077d97821c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:32.393573 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efadd5-5c6e-4666-a9cf-26077d97821c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:32.393951 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efadd5-5c6e-4666-a9cf-26077d97821c None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:32.394944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22efadd5-5c6e-4666-a9cf-26077d97821c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:32.397345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22efadd5-5c6e-4666-a9cf-26077d97821c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:32.397686 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22efadd5-5c6e-4666-a9cf-26077d97821c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:32.413915 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ae675-8a61-4305-a6ac-b90983ab64a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:32.415347 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ae675-8a61-4305-a6ac-b90983ab64a4 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:32.417060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ae675-8a61-4305-a6ac-b90983ab64a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:32.417792 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ae675-8a61-4305-a6ac-b90983ab64a4 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:32.419554 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ae675-8a61-4305-a6ac-b90983ab64a4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:32.420307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ae675-8a61-4305-a6ac-b90983ab64a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:32.423136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ae675-8a61-4305-a6ac-b90983ab64a4 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:32.425488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 403/803] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Sep 6 16:01:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:32.427331 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efadd5-5c6e-4666-a9cf-26077d97821c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:32.435637 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 401/804] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 6 16:01:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:32.438479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ae675-8a61-4305-a6ac-b90983ab64a4 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:32.608395 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ae675-8a61-4305-a6ac-b90983ab64a4 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:32.642943 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efadd5-5c6e-4666-a9cf-26077d97821c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:32.657097 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc1359-80e1-4d29-8fc0-3f96dbf8f697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:32.659194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc1359-80e1-4d29-8fc0-3f96dbf8f697 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:32.659930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc1359-80e1-4d29-8fc0-3f96dbf8f697 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:32.661776 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc1359-80e1-4d29-8fc0-3f96dbf8f697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:32.689150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc1359-80e1-4d29-8fc0-3f96dbf8f697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:32.689918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb98e00-2ab1-407e-82d0-e976bb4520c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:32.690739 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb98e00-2ab1-407e-82d0-e976bb4520c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:32.691159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb98e00-2ab1-407e-82d0-e976bb4520c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:32.691531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb98e00-2ab1-407e-82d0-e976bb4520c6 None admin] PATH_INFO: `/v3/projects/0138d62d5879495499a17e20310d4f61/users/ba58a4914ddf418db9e8b9dce5784cf8/roles/56d621cf28924933b6356795a6f8ba23` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:32.694047 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc1359-80e1-4d29-8fc0-3f96dbf8f697 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:32.710472 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:32.720179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb98e00-2ab1-407e-82d0-e976bb4520c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=0138d62d5879495499a17e20310d4f61, user_id=ba58a4914ddf418db9e8b9dce5784cf8, role_id=56d621cf28924933b6356795a6f8ba23)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:32.720596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-fccc1359-80e1-4d29-8fc0-3f96dbf8f697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:32.720596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:32.720596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:32.720596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:32.720596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:32.720596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:32.720596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:32.720596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:32.720596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:32.720596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:32.720596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:32.720596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:32.720596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:32.720596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:32.720596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:32.720596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:32.722374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:32.722374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:32.722374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:32.722374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:32.722374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:32.722374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:32.722374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:32.722374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:32.722374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:32.722374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:32.722374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:32.722374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:32.722374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:32.722374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:32.722374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:32.722374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:32.722374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:32.724757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:32.724757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:32.724757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:32.727412 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 402/805] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:32.728518 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc1359-80e1-4d29-8fc0-3f96dbf8f697 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:32.729467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb98e00-2ab1-407e-82d0-e976bb4520c6 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:32.732114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:32.764001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 404/806] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:01:32 2022] PUT /identity/v3/projects/0138d62d5879495499a17e20310d4f61/users/ba58a4914ddf418db9e8b9dce5784cf8/roles/56d621cf28924933b6356795a6f8ba23 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:01:32.767451 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb98e00-2ab1-407e-82d0-e976bb4520c6 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:33.021190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc1359-80e1-4d29-8fc0-3f96dbf8f697 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:33.024373 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb98e00-2ab1-407e-82d0-e976bb4520c6 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:33.064569 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79fdbc-0f4a-420f-86ac-44d8f8b92422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:33.065187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79fdbc-0f4a-420f-86ac-44d8f8b92422 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:33.065466 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79fdbc-0f4a-420f-86ac-44d8f8b92422 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:33.065738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79fdbc-0f4a-420f-86ac-44d8f8b92422 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:33.066383 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79fdbc-0f4a-420f-86ac-44d8f8b92422 None admin] RBAC: Adding query filter params (type=network) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:33.074705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79fdbc-0f4a-420f-86ac-44d8f8b92422 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:33.074977 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a2609-cf49-4009-976a-d4925a8823a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:33.074977 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a2609-cf49-4009-976a-d4925a8823a7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:33.074977 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a2609-cf49-4009-976a-d4925a8823a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:33.075355 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79fdbc-0f4a-420f-86ac-44d8f8b92422 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:33.075511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a2609-cf49-4009-976a-d4925a8823a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:33.075511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a2609-cf49-4009-976a-d4925a8823a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:33.075511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a2609-cf49-4009-976a-d4925a8823a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:33.075511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a2609-cf49-4009-976a-d4925a8823a7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:33.075961 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 403/807] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 6 16:01:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:33.075961 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f77ad-c0a1-46b3-8030-3ffa9c6021d9 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:33.076326 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f77ad-c0a1-46b3-8030-3ffa9c6021d9 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:33.076326 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f77ad-c0a1-46b3-8030-3ffa9c6021d9 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:33.081929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 404/808] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:33.083859 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cccdc0-d908-42f7-a40c-901e34c31796 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:33.084136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cccdc0-d908-42f7-a40c-901e34c31796 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:33.084433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cccdc0-d908-42f7-a40c-901e34c31796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:33.085391 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 405/809] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:33.086258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a2609-cf49-4009-976a-d4925a8823a7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:33.115049 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-51cccdc0-d908-42f7-a40c-901e34c31796 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:33.121777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-51cccdc0-d908-42f7-a40c-901e34c31796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:33.149016 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:33.214711 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:33.221611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a2609-cf49-4009-976a-d4925a8823a7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:33.233613 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 405/810] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:33.234923 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cccdc0-d908-42f7-a40c-901e34c31796 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:33.358769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12595e-3568-422f-a833-edd08f566baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:33.360180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12595e-3568-422f-a833-edd08f566baf None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:33.361516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12595e-3568-422f-a833-edd08f566baf None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:33.362144 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12595e-3568-422f-a833-edd08f566baf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:33.364001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12595e-3568-422f-a833-edd08f566baf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:33.371072 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12595e-3568-422f-a833-edd08f566baf None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:33.483394 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 406/811] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:01:33 2022] POST /identity/v3/endpoints => generated 337 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:01:33.488016 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e075d-0fc7-4746-9e5e-c6a93b3bde1b None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:33.494699 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e075d-0fc7-4746-9e5e-c6a93b3bde1b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:33.496735 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e075d-0fc7-4746-9e5e-c6a93b3bde1b None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:33.502916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 407/812] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:33 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:33.515743 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49735af9-28b4-4324-a5b7-a3f776fe80a5 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:33.516655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49735af9-28b4-4324-a5b7-a3f776fe80a5 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:33.517567 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49735af9-28b4-4324-a5b7-a3f776fe80a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:33.556011 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-49735af9-28b4-4324-a5b7-a3f776fe80a5 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:33.557480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-49735af9-28b4-4324-a5b7-a3f776fe80a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:33.569335 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cccdc0-d908-42f7-a40c-901e34c31796 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:33.653754 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276812a-eefc-4a6d-878f-d79aa01b90c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:33.655306 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276812a-eefc-4a6d-878f-d79aa01b90c9 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:33.656054 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276812a-eefc-4a6d-878f-d79aa01b90c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:33.656846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276812a-eefc-4a6d-878f-d79aa01b90c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:33.666702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 408/813] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:33.666702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49735af9-28b4-4324-a5b7-a3f776fe80a5 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:33.688085 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276812a-eefc-4a6d-878f-d79aa01b90c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:33.691346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276812a-eefc-4a6d-878f-d79aa01b90c9 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:33.713380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-a276812a-eefc-4a6d-878f-d79aa01b90c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:33.713380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:33.713380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:33.713380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:33.713380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:33.713380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:33.713380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:33.713380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:33.713380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:33.713380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:33.713380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:33.713380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:33.713380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:33.713380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:33.713380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:33.714510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:33.714510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:33.714510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:33.714510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:33.714510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:33.714510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:33.714510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:33.714510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:33.714510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:33.714510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:33.714510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:33.714510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:33.714510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:33.714510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:33.714510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:33.714510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:33.714510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:33.715618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:33.715618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:33.715618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:33.715618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:33.715618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:33.715618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:33.715618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:33.715618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:33.717965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 406/814] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 483 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:33.725408 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276812a-eefc-4a6d-878f-d79aa01b90c9 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:33.729123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:33.801469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:33.854315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49735af9-28b4-4324-a5b7-a3f776fe80a5 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:33.876745 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276812a-eefc-4a6d-878f-d79aa01b90c9 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:33.931757 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abf008-3bcc-4793-83c5-91d156175ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:33.934236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c829bf2-9a7f-4ecd-bad7-0f1d69e4ecc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:33.935806 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abf008-3bcc-4793-83c5-91d156175ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:33.937103 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c829bf2-9a7f-4ecd-bad7-0f1d69e4ecc1 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:33.937886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abf008-3bcc-4793-83c5-91d156175ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:33.938576 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c829bf2-9a7f-4ecd-bad7-0f1d69e4ecc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:33.939316 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abf008-3bcc-4793-83c5-91d156175ba5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:33.940029 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c829bf2-9a7f-4ecd-bad7-0f1d69e4ecc1 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:33.943651 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c829bf2-9a7f-4ecd-bad7-0f1d69e4ecc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:33.944521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c829bf2-9a7f-4ecd-bad7-0f1d69e4ecc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:33.948271 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c829bf2-9a7f-4ecd-bad7-0f1d69e4ecc1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:33.967192 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abf008-3bcc-4793-83c5-91d156175ba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:01:33.974188 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 407/815] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:33.977734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-e8abf008-3bcc-4793-83c5-91d156175ba5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:01:33.977734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:33.977734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:33.977734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:33.977734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:33.977734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:33.977734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:33.977734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:33.977734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:33.977734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:33.977734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:33.977734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:33.977734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:01:33.977734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:01:33.977734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:01:33.977734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:01:33.979332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:01:33.979332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:01:33.979332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:01:33.979332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:01:33.979332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:33.979332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:33.979332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:33.979332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:33.979332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:33.979332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:33.979332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:33.979332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:33.979332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:33.979332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:33.979332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:33.979332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:33.980705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:33.980705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:33.980705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:33.980705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:33.980705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:01:33.980705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:01:33.980705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:01:33.980705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:01:33.980705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:01:33.980705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:01:33.980705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:01:33.980705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:33.980705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 409/816] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:01:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:33.981899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c829bf2-9a7f-4ecd-bad7-0f1d69e4ecc1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:33.986957 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abf008-3bcc-4793-83c5-91d156175ba5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:34.102794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:34.107840 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abf008-3bcc-4793-83c5-91d156175ba5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:34.123824 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c829bf2-9a7f-4ecd-bad7-0f1d69e4ecc1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:34.140419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:34.156684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4387b-cc07-4571-a986-b51a28afc746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:34.157379 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4387b-cc07-4571-a986-b51a28afc746 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:34.157925 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4387b-cc07-4571-a986-b51a28afc746 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:34.158205 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4387b-cc07-4571-a986-b51a28afc746 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:34.158932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4387b-cc07-4571-a986-b51a28afc746 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:34.159384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4387b-cc07-4571-a986-b51a28afc746 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:34.160904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4387b-cc07-4571-a986-b51a28afc746 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:34.180123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 410/817] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:01:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:34.182016 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4387b-cc07-4571-a986-b51a28afc746 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:34.185464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5138f-bee7-4997-97af-85d0482a8231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:34.186165 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5138f-bee7-4997-97af-85d0482a8231 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:34.186455 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5138f-bee7-4997-97af-85d0482a8231 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:34.186781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5138f-bee7-4997-97af-85d0482a8231 None admin] PATH_INFO: `/v3/projects/0138d62d5879495499a17e20310d4f61/users/ece176d412574790a13d03dcca08f91a/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:34.188806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5138f-bee7-4997-97af-85d0482a8231 None admin] RBAC: Authorizing `identity:create_grant(project_id=0138d62d5879495499a17e20310d4f61, user_id=ece176d412574790a13d03dcca08f91a, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:34.190247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5138f-bee7-4997-97af-85d0482a8231 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:34.208996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 408/818] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:01:33 2022] PUT /identity/v3/projects/0138d62d5879495499a17e20310d4f61/users/ece176d412574790a13d03dcca08f91a/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:01:34.210370 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5138f-bee7-4997-97af-85d0482a8231 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:34.390469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4387b-cc07-4571-a986-b51a28afc746 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:34.435185 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5138f-bee7-4997-97af-85d0482a8231 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:34.446597 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f850d-98d9-46fd-94b8-7e5c3a0a84fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:34.448229 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f850d-98d9-46fd-94b8-7e5c3a0a84fd None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:34.448712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f850d-98d9-46fd-94b8-7e5c3a0a84fd None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:34.449174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f850d-98d9-46fd-94b8-7e5c3a0a84fd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:34.450357 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f850d-98d9-46fd-94b8-7e5c3a0a84fd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:34.452860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f850d-98d9-46fd-94b8-7e5c3a0a84fd None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:34.466326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-fa7f850d-98d9-46fd-94b8-7e5c3a0a84fd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 16:01:34.466326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:34.466326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:01:34.466326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:01:34.466326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:01:34.466326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:01:34.466326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:01:34.466326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:34.466326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 16:01:34.466326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:34.466326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:01:34.466326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:34.466326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:34.466326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:34.466326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:34.466326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:34.466326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:34.467620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:34.467620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:34.467620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:34.467620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:34.467620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:34.467620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:34.467620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:01:34.467620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:01:34.467620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:01:34.467620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:01:34.467620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:34.467620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:34.467620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:34.467620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:34.467620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:34.467620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:34.468437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:34.468437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:34.468437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:34.468437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:34.468437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:34.468437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:34.468437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:34.468437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:34.468437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:34.468437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:34.468437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:01:34.468437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:34.468437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 16:01:34.468437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:34.470792 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 411/819] 10.222.0.26 () {56 vars in 1129 bytes} [Tue Sep 6 16:01:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:34.482303 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f850d-98d9-46fd-94b8-7e5c3a0a84fd None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:34.495340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ef943-ed3d-4d98-ac71-4770568951cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:34.496349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ef943-ed3d-4d98-ac71-4770568951cd None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:34.496818 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ef943-ed3d-4d98-ac71-4770568951cd None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:34.497278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ef943-ed3d-4d98-ac71-4770568951cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:34.517365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:34.517840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ef943-ed3d-4d98-ac71-4770568951cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:34.521247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ef943-ed3d-4d98-ac71-4770568951cd None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:34.554932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-905ef943-ed3d-4d98-ac71-4770568951cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:34.554932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:34.554932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:34.554932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:34.554932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:34.554932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:34.554932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:34.554932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:34.554932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:34.554932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:34.554932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:34.554932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:34.554932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:34.554932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:34.554932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:34.554932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:34.556814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:34.556814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:34.556814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:34.556814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:34.556814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:34.556814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:34.556814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:34.556814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:34.556814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:34.556814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:34.556814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:34.556814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:34.556814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:34.556814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:34.556814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:34.556814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:34.556814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:34.558463 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:34.558463 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:34.558463 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:34.560619 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 409/820] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:34.582734 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ef943-ed3d-4d98-ac71-4770568951cd None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:34.616812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:34.678741 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f850d-98d9-46fd-94b8-7e5c3a0a84fd None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:34.701134 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ef943-ed3d-4d98-ac71-4770568951cd None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:34.739580 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c5fd2-b605-476a-8bee-b6de0f8fec5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:34.740370 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c5fd2-b605-476a-8bee-b6de0f8fec5a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:34.740769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c5fd2-b605-476a-8bee-b6de0f8fec5a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:34.741150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c5fd2-b605-476a-8bee-b6de0f8fec5a None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:34.741907 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c5fd2-b605-476a-8bee-b6de0f8fec5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:34.742319 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c5fd2-b605-476a-8bee-b6de0f8fec5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:34.743842 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c5fd2-b605-476a-8bee-b6de0f8fec5a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:34.745662 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345b1e3-45f7-4c57-a0c3-feeeb3da2d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:34.747126 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345b1e3-45f7-4c57-a0c3-feeeb3da2d40 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:34.747627 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345b1e3-45f7-4c57-a0c3-feeeb3da2d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:34.748115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345b1e3-45f7-4c57-a0c3-feeeb3da2d40 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:34.749034 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345b1e3-45f7-4c57-a0c3-feeeb3da2d40 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:34.749587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345b1e3-45f7-4c57-a0c3-feeeb3da2d40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:34.752018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345b1e3-45f7-4c57-a0c3-feeeb3da2d40 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:34.761639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 412/822] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Sep 6 16:01:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:34.762932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 410/822] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:34.764719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345b1e3-45f7-4c57-a0c3-feeeb3da2d40 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:34.767329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c5fd2-b605-476a-8bee-b6de0f8fec5a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:34.870110 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:34.889928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c5fd2-b605-476a-8bee-b6de0f8fec5a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:34.931710 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce09205-384e-4cd3-b4f7-c96890846c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:34.932749 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce09205-384e-4cd3-b4f7-c96890846c29 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:34.933284 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce09205-384e-4cd3-b4f7-c96890846c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:34.933767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce09205-384e-4cd3-b4f7-c96890846c29 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:34.934635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce09205-384e-4cd3-b4f7-c96890846c29 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:34.935186 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce09205-384e-4cd3-b4f7-c96890846c29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:34.936912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce09205-384e-4cd3-b4f7-c96890846c29 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:34.943808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 411/823] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Sep 6 16:01:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:34.949282 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce09205-384e-4cd3-b4f7-c96890846c29 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:34.967508 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:34.977710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345b1e3-45f7-4c57-a0c3-feeeb3da2d40 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:35.024015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c93b2d-8f7d-45a3-b078-322572f2fa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:35.025130 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c93b2d-8f7d-45a3-b078-322572f2fa63 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:35.025630 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c93b2d-8f7d-45a3-b078-322572f2fa63 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:35.026327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c93b2d-8f7d-45a3-b078-322572f2fa63 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:35.133823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce09205-384e-4cd3-b4f7-c96890846c29 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:35.174870 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb56ec-588d-41b4-b93c-8a456b063936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:35.176171 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb56ec-588d-41b4-b93c-8a456b063936 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:35.176725 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb56ec-588d-41b4-b93c-8a456b063936 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:35.177206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb56ec-588d-41b4-b93c-8a456b063936 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:35.178096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb56ec-588d-41b4-b93c-8a456b063936 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:35.179722 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb56ec-588d-41b4-b93c-8a456b063936 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:35.184814 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c93b2d-8f7d-45a3-b078-322572f2fa63 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:35.187518 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c93b2d-8f7d-45a3-b078-322572f2fa63 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:35.209393 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 412/824] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:01:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:01:35.218847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-f8c93b2d-8f7d-45a3-b078-322572f2fa63 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:01:35.218847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:35.218847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:35.218847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:35.218847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:35.218847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:35.218847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:35.218847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:35.218847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:35.218847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:35.218847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:35.218847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:35.218847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:35.218847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:35.218847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:35.218847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:35.219861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:35.219861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:35.219861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:35.219861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:35.219861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:35.219861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:35.219861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:35.219861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:35.219861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:35.219861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:35.219861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:35.219861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:35.219861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:35.219861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:35.219861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:35.219861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:35.219861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:35.219861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:35.220864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:35.220864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:35.220864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:35.220864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:35.220864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:35.220864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:35.220864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:35.220864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:35.220864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:01:35.220864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:35.220864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 413/825] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Sep 6 16:01:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 449 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:35.224128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb56ec-588d-41b4-b93c-8a456b063936 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:35.230044 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:35.369808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb56ec-588d-41b4-b93c-8a456b063936 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:35.412367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88965110-1563-4dbd-9f25-1414f9029587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:35.413373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88965110-1563-4dbd-9f25-1414f9029587 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:35.413900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88965110-1563-4dbd-9f25-1414f9029587 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:35.414392 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88965110-1563-4dbd-9f25-1414f9029587 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:35.415324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88965110-1563-4dbd-9f25-1414f9029587 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:35.415786 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88965110-1563-4dbd-9f25-1414f9029587 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:35.417411 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88965110-1563-4dbd-9f25-1414f9029587 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:35.439587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 413/826] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Sep 6 16:01:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:35.445253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c93b2d-8f7d-45a3-b078-322572f2fa63 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:35.600131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c93b2d-8f7d-45a3-b078-322572f2fa63 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:35.605124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18db36-6687-492c-8e88-2c5e787d7467 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:35.605124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18db36-6687-492c-8e88-2c5e787d7467 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:35.605124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18db36-6687-492c-8e88-2c5e787d7467 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:35.612837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 414/827] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:35.615895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293600b-04b7-4e8d-8976-8f70941e63a8 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:35.616355 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293600b-04b7-4e8d-8976-8f70941e63a8 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:35.616807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293600b-04b7-4e8d-8976-8f70941e63a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:35.642359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bddb9-2324-4dba-a757-c61ded345dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:35.643534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bddb9-2324-4dba-a757-c61ded345dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:35.643969 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bddb9-2324-4dba-a757-c61ded345dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:35.644393 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bddb9-2324-4dba-a757-c61ded345dc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:35.649432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-1293600b-04b7-4e8d-8976-8f70941e63a8 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:35.650894 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1293600b-04b7-4e8d-8976-8f70941e63a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:35.667246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:35.743298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:35.758451 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46bddb9-2324-4dba-a757-c61ded345dc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:35.761404 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46bddb9-2324-4dba-a757-c61ded345dc1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:35.763777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 415/828] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:35.881539 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 414/829] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:01:35 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:01:35.887827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293600b-04b7-4e8d-8976-8f70941e63a8 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:36.018957 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293600b-04b7-4e8d-8976-8f70941e63a8 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:36.080861 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58034610-cb75-4d8f-97d1-8194e95c218d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:36.081928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58034610-cb75-4d8f-97d1-8194e95c218d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:36.082419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58034610-cb75-4d8f-97d1-8194e95c218d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:36.083548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58034610-cb75-4d8f-97d1-8194e95c218d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:36.102011 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58034610-cb75-4d8f-97d1-8194e95c218d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:36.103664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58034610-cb75-4d8f-97d1-8194e95c218d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:36.126487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-58034610-cb75-4d8f-97d1-8194e95c218d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:36.126487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:36.126487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:36.126487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:36.126487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:36.126487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:36.126487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:36.126487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:36.126487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:36.126487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:36.126487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:36.126487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:36.126487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:36.126487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:36.126487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:36.126487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:36.127532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:36.127532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:36.127532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:36.127532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:36.127532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:36.127532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:36.127532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:36.127532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:36.127532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:36.127532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:36.127532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:36.127532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:36.127532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:36.127532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:36.127532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:36.127532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:36.127532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:36.128441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:36.128441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:36.128441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:36.128441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 416/830] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:36.135006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bddb9-2324-4dba-a757-c61ded345dc1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:36.138070 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58034610-cb75-4d8f-97d1-8194e95c218d None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:36.193983 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:36.200879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:36.279823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58034610-cb75-4d8f-97d1-8194e95c218d None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:36.288437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bddb9-2324-4dba-a757-c61ded345dc1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:36.323615 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f35ac-2536-445b-b8e9-08fe30a2ec0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:36.324749 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f35ac-2536-445b-b8e9-08fe30a2ec0e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:36.325236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f35ac-2536-445b-b8e9-08fe30a2ec0e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:36.325788 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f35ac-2536-445b-b8e9-08fe30a2ec0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:36.339126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f35ac-2536-445b-b8e9-08fe30a2ec0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:01:36.347782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-598f35ac-2536-445b-b8e9-08fe30a2ec0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:01:36.347782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:36.347782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:36.347782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:36.347782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:36.347782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:36.347782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:36.347782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:36.347782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:36.347782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:36.347782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:36.347782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:36.347782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:01:36.347782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:01:36.347782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:01:36.347782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:01:36.348905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:01:36.348905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:01:36.348905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:01:36.348905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:01:36.348905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:36.348905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:36.348905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:36.348905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:36.348905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:36.348905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:36.348905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:36.348905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:36.348905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:36.348905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:36.348905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:36.348905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:36.349743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:36.349743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:36.349743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:36.349743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:36.349743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:01:36.349743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:01:36.349743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:01:36.349743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:01:36.349743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:01:36.349743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:01:36.349743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:01:36.349743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:36.352324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 417/831] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:01:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:36.361146 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e0e48-828c-4857-8e1f-d1a2fdf18a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:36.361777 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e0e48-828c-4857-8e1f-d1a2fdf18a3b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:36.362857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e0e48-828c-4857-8e1f-d1a2fdf18a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:36.363164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f35ac-2536-445b-b8e9-08fe30a2ec0e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:36.364171 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e0e48-828c-4857-8e1f-d1a2fdf18a3b None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:36.365732 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e0e48-828c-4857-8e1f-d1a2fdf18a3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:36.366465 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e0e48-828c-4857-8e1f-d1a2fdf18a3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:36.369401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e0e48-828c-4857-8e1f-d1a2fdf18a3b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:36.393355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 415/832] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:36.399579 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e0e48-828c-4857-8e1f-d1a2fdf18a3b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:36.513252 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:36.519426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f35ac-2536-445b-b8e9-08fe30a2ec0e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:36.596585 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1fa75-7b37-4bf6-8c81-6c3c86f383ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:36.598178 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1fa75-7b37-4bf6-8c81-6c3c86f383ef None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:36.599610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1fa75-7b37-4bf6-8c81-6c3c86f383ef None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:36.600387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1fa75-7b37-4bf6-8c81-6c3c86f383ef None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:36.601779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1fa75-7b37-4bf6-8c81-6c3c86f383ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:36.602625 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1fa75-7b37-4bf6-8c81-6c3c86f383ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:36.606640 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1fa75-7b37-4bf6-8c81-6c3c86f383ef None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:36.622001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e0e48-828c-4857-8e1f-d1a2fdf18a3b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:36.630335 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 418/833] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:01:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:36.633018 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37264dfc-f51f-41a6-8f49-69b01b3fa55c None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:36.633784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37264dfc-f51f-41a6-8f49-69b01b3fa55c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:36.634541 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37264dfc-f51f-41a6-8f49-69b01b3fa55c None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:36.637645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 419/834] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:36.640252 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37264dfc-f51f-41a6-8f49-69b01b3fa55c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:36.642223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:36.696975 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5d1e4-ce2b-47a6-9712-c2ebac45e52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:36.697842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5d1e4-ce2b-47a6-9712-c2ebac45e52e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:36.698303 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5d1e4-ce2b-47a6-9712-c2ebac45e52e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:36.698803 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5d1e4-ce2b-47a6-9712-c2ebac45e52e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:36.710652 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5d1e4-ce2b-47a6-9712-c2ebac45e52e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:36.713212 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5d1e4-ce2b-47a6-9712-c2ebac45e52e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:36.728700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-28e5d1e4-ce2b-47a6-9712-c2ebac45e52e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:36.728700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:36.728700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:36.728700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:36.728700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:36.728700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:36.728700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:36.728700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:36.728700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:36.728700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:36.728700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:36.728700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:36.728700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:36.728700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:36.728700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:36.730384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:36.730384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:36.730384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:36.730384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:36.730384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:36.730384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:36.730384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:36.730384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:36.730384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:36.730384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:36.730384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:36.730384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:36.730384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:36.730384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:36.730384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:36.730384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:36.730384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:36.732730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:36.732730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:36.732730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:36.732730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:36.732730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:36.732730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:36.732730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:36.732730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:36.732730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 416/835] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:36.732730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160370f-608d-49f8-a6ff-073548f48141 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:36.732730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160370f-608d-49f8-a6ff-073548f48141 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:36.733893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160370f-608d-49f8-a6ff-073548f48141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:36.752813 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0160370f-608d-49f8-a6ff-073548f48141 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:36.754123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0160370f-608d-49f8-a6ff-073548f48141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:36.813866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37264dfc-f51f-41a6-8f49-69b01b3fa55c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:36.854884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730f814-7c74-4a84-96ab-c7be037dd79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:36.855525 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730f814-7c74-4a84-96ab-c7be037dd79b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:36.855941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730f814-7c74-4a84-96ab-c7be037dd79b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:36.856224 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730f814-7c74-4a84-96ab-c7be037dd79b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:36.869194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730f814-7c74-4a84-96ab-c7be037dd79b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:36.869445 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 417/836] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:36.870464 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0160370f-608d-49f8-a6ff-073548f48141 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:36.886968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730f814-7c74-4a84-96ab-c7be037dd79b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:36.903346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-4730f814-7c74-4a84-96ab-c7be037dd79b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:36.903346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:36.903346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:36.903346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:36.903346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:36.903346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:36.903346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:36.903346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:36.903346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:36.903346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:36.903346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:36.903346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:36.903346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:36.903346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:36.903346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:36.903346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:36.904526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:36.904526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:36.904526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:36.904526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:36.904526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:36.904526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:36.904526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:36.904526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:36.904526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:36.904526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:36.904526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:36.904526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:36.904526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:36.904526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:36.904526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:36.904526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:36.904526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:36.905415 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:36.905415 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:36.905415 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:36.912190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 420/837] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:36.913788 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730f814-7c74-4a84-96ab-c7be037dd79b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:36.933386 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:37.031449 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:37.038943 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0160370f-608d-49f8-a6ff-073548f48141 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:37.056012 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730f814-7c74-4a84-96ab-c7be037dd79b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:37.096674 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40dc8c-0a57-494b-a57a-6ebd8c7b56f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:37.097746 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40dc8c-0a57-494b-a57a-6ebd8c7b56f6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:37.098334 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40dc8c-0a57-494b-a57a-6ebd8c7b56f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:37.098947 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40dc8c-0a57-494b-a57a-6ebd8c7b56f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:37.100504 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40dc8c-0a57-494b-a57a-6ebd8c7b56f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:37.100999 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40dc8c-0a57-494b-a57a-6ebd8c7b56f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:37.102881 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40dc8c-0a57-494b-a57a-6ebd8c7b56f6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:37.103359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73080d-011b-4416-9164-5d83f6ef901c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:37.104396 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73080d-011b-4416-9164-5d83f6ef901c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:37.104893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73080d-011b-4416-9164-5d83f6ef901c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:37.105510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73080d-011b-4416-9164-5d83f6ef901c None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:37.107101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73080d-011b-4416-9164-5d83f6ef901c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:37.107240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73080d-011b-4416-9164-5d83f6ef901c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:37.109836 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73080d-011b-4416-9164-5d83f6ef901c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:37.123270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 421/838] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:37.124985 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40dc8c-0a57-494b-a57a-6ebd8c7b56f6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:37.139109 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 418/839] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:37.142190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73080d-011b-4416-9164-5d83f6ef901c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:37.216171 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:37.246276 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40dc8c-0a57-494b-a57a-6ebd8c7b56f6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:37.303529 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b473d8-34b5-43c8-b4b4-68691bd78245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:37.304529 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b473d8-34b5-43c8-b4b4-68691bd78245 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:37.305047 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b473d8-34b5-43c8-b4b4-68691bd78245 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:37.305261 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b473d8-34b5-43c8-b4b4-68691bd78245 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:37.320827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b473d8-34b5-43c8-b4b4-68691bd78245 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:37.324548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b473d8-34b5-43c8-b4b4-68691bd78245 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:37.327396 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73080d-011b-4416-9164-5d83f6ef901c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:37.341112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-c5b473d8-34b5-43c8-b4b4-68691bd78245 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:01:37.341112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:37.341112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:37.341112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:37.341112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:37.341112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:37.341112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:37.341112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:37.341112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:37.341112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:37.341112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:37.341112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:37.341112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:37.341112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:37.341112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:37.341112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:37.342138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:37.342138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:37.342138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:37.342138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:37.342138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:37.342138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:37.342138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:37.342138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:37.342138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:37.342138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:37.342138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:37.342138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:37.342138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:37.342138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:37.342138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:37.342138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:37.342138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:37.342138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:37.343242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:37.343242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:37.343242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:37.343242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:37.343242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:37.343242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:37.343242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:37.343242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:37.343242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:01:37.343242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:37.343242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 422/840] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 6 16:01:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:37.343242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b473d8-34b5-43c8-b4b4-68691bd78245 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:37.391294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e214af0-d94e-40c5-a2ef-695ee635615f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:37.391909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e214af0-d94e-40c5-a2ef-695ee635615f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:37.392249 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e214af0-d94e-40c5-a2ef-695ee635615f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:37.392917 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e214af0-d94e-40c5-a2ef-695ee635615f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:37.393254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e214af0-d94e-40c5-a2ef-695ee635615f None admin] RBAC: Adding query filter params (role.id=ba05677650e046a48d5f19440d48fbf2, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=6afd4f6ffdae4b16aa5ba757e68c18c4) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:37.393560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e214af0-d94e-40c5-a2ef-695ee635615f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:37.395027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e214af0-d94e-40c5-a2ef-695ee635615f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:37.410265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 419/841] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:01:37 2022] GET /identity/v3/role_assignments?user.id=6afd4f6ffdae4b16aa5ba757e68c18c4&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:37.410265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e214af0-d94e-40c5-a2ef-695ee635615f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:37.447137 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:37.552436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b473d8-34b5-43c8-b4b4-68691bd78245 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:37.597423 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e214af0-d94e-40c5-a2ef-695ee635615f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:37.623071 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2491b-54cd-4bb3-b327-ba5036062f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:37.624398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2491b-54cd-4bb3-b327-ba5036062f7f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:37.624682 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2491b-54cd-4bb3-b327-ba5036062f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:37.624953 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2491b-54cd-4bb3-b327-ba5036062f7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:37.627656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:37.644078 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2491b-54cd-4bb3-b327-ba5036062f7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:01:37.650410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-fce2491b-54cd-4bb3-b327-ba5036062f7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:01:37.650410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:37.650410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:37.650410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:37.650410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:37.650410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:37.650410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:37.650410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:37.650410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:37.650410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:37.650410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:37.650410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:37.650410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:01:37.650410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:01:37.650410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:01:37.650410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:01:37.651630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:01:37.651630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:01:37.651630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:01:37.651630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:01:37.651630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:37.651630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:37.651630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:37.651630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:37.651630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:37.651630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:37.651630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:37.651630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:37.651630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:37.651630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:37.651630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:37.651630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:37.652479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:37.652479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:37.652479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:37.652479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:37.652479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:01:37.652479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:01:37.652479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:01:37.652479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:01:37.652479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:01:37.652479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:01:37.652479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:01:37.652479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:37.653163 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 423/842] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:01:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:37.653899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae1ee8-02fe-4289-8259-dd324ef2f2d8 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:37.654180 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae1ee8-02fe-4289-8259-dd324ef2f2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:37.654453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae1ee8-02fe-4289-8259-dd324ef2f2d8 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:37.656082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 424/843] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:37.662881 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28564088-9ef1-4a66-890c-012a48313651 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:37.663202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28564088-9ef1-4a66-890c-012a48313651 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:37.663432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28564088-9ef1-4a66-890c-012a48313651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:37.667414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780cb1e4-1c8c-471f-a3e3-fdcc6cb7bd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:37.668064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cb1e4-1c8c-471f-a3e3-fdcc6cb7bd4f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:37.668336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cb1e4-1c8c-471f-a3e3-fdcc6cb7bd4f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:37.668617 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cb1e4-1c8c-471f-a3e3-fdcc6cb7bd4f None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:37.669377 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780cb1e4-1c8c-471f-a3e3-fdcc6cb7bd4f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:37.669688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780cb1e4-1c8c-471f-a3e3-fdcc6cb7bd4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:37.671215 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780cb1e4-1c8c-471f-a3e3-fdcc6cb7bd4f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:37.687494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-28564088-9ef1-4a66-890c-012a48313651 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:37.689111 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-28564088-9ef1-4a66-890c-012a48313651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:37.695318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 420/844] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Sep 6 16:01:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:37.696499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780cb1e4-1c8c-471f-a3e3-fdcc6cb7bd4f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:37.779695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 425/845] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:37.780584 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28564088-9ef1-4a66-890c-012a48313651 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:37.874160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:37.898094 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780cb1e4-1c8c-471f-a3e3-fdcc6cb7bd4f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:37.937615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44069119-6642-4b60-8ea0-429045bd421b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:37.938551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44069119-6642-4b60-8ea0-429045bd421b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:37.939179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44069119-6642-4b60-8ea0-429045bd421b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:37.939670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44069119-6642-4b60-8ea0-429045bd421b None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:37.940595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44069119-6642-4b60-8ea0-429045bd421b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:37.941114 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44069119-6642-4b60-8ea0-429045bd421b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:37.942803 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44069119-6642-4b60-8ea0-429045bd421b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:37.957923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 421/846] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:01:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:37.959616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44069119-6642-4b60-8ea0-429045bd421b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:37.975433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28564088-9ef1-4a66-890c-012a48313651 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:38.022786 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:38.044032 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bc3f5-b6ec-4e1e-abab-1324f535b7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:38.045604 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bc3f5-b6ec-4e1e-abab-1324f535b7bf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:38.045791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bc3f5-b6ec-4e1e-abab-1324f535b7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:38.046113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bc3f5-b6ec-4e1e-abab-1324f535b7bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:38.157400 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3bc3f5-b6ec-4e1e-abab-1324f535b7bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:38.159189 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3bc3f5-b6ec-4e1e-abab-1324f535b7bf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:38.163492 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44069119-6642-4b60-8ea0-429045bd421b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:38.207881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34490b3b-7355-4fd8-83bd-f5fd11aab2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:38.208876 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34490b3b-7355-4fd8-83bd-f5fd11aab2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:38.209328 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34490b3b-7355-4fd8-83bd-f5fd11aab2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:38.209776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34490b3b-7355-4fd8-83bd-f5fd11aab2f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:38.222292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34490b3b-7355-4fd8-83bd-f5fd11aab2f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:38.224356 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34490b3b-7355-4fd8-83bd-f5fd11aab2f7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:38.227552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:38.246176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-34490b3b-7355-4fd8-83bd-f5fd11aab2f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:38.246176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:38.246176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:38.246176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:38.246176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:38.246176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:38.246176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:38.246176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:38.246176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:38.246176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:38.246176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:38.246176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:38.246176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:38.246176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:38.246176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:38.246176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:38.247247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:38.247247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:38.247247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:38.247247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:38.247247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:38.247247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:38.247247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:38.247247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:38.247247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:38.247247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:38.247247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:38.247247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:38.247247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:38.247247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:38.247247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:38.247247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:38.247247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:38.248157 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:38.248157 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:38.248157 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:38.248157 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 422/847] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:38.248157 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34490b3b-7355-4fd8-83bd-f5fd11aab2f7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:38.317766 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 426/848] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:01:37 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:01:38.319851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bc3f5-b6ec-4e1e-abab-1324f535b7bf None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:38.368977 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:38.401544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34490b3b-7355-4fd8-83bd-f5fd11aab2f7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:38.457620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd6c8d-2064-4081-88c5-aa7fd462caef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:38.458956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd6c8d-2064-4081-88c5-aa7fd462caef None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:38.459374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd6c8d-2064-4081-88c5-aa7fd462caef None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:38.459923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd6c8d-2064-4081-88c5-aa7fd462caef None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:38.461593 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd6c8d-2064-4081-88c5-aa7fd462caef None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:38.463211 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd6c8d-2064-4081-88c5-aa7fd462caef None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:38.474433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-eadd6c8d-2064-4081-88c5-aa7fd462caef None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 16:01:38.474433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:38.474433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:01:38.474433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:01:38.474433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:01:38.474433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:01:38.474433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:01:38.474433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:38.474433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 16:01:38.474433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:38.474433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:01:38.474433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:38.474433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:38.474433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:38.474433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:38.474433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:38.474433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:38.476405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:38.476405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:38.476405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:38.476405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:38.476405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:38.476405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:38.476405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:01:38.476405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:01:38.476405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:01:38.476405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:01:38.476405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:38.476405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:38.476405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:38.476405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:38.476405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:38.476405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:38.478857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:38.478857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:38.478857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:38.478857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:38.478857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:38.478857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:38.478857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:38.478857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:38.478857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:38.478857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:38.478857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:01:38.478857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:38.478857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 16:01:38.478857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:38.478857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 423/849] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 16:01:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:38.481948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd6c8d-2064-4081-88c5-aa7fd462caef None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:38.515912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bc3f5-b6ec-4e1e-abab-1324f535b7bf None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:38.568476 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7597471-0ec0-48fa-b32a-f52fd71c86ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:38.569739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7597471-0ec0-48fa-b32a-f52fd71c86ac None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:38.570132 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7597471-0ec0-48fa-b32a-f52fd71c86ac None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:38.570465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7597471-0ec0-48fa-b32a-f52fd71c86ac None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:38.571232 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7597471-0ec0-48fa-b32a-f52fd71c86ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:38.571589 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7597471-0ec0-48fa-b32a-f52fd71c86ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:38.573379 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7597471-0ec0-48fa-b32a-f52fd71c86ac None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:38.588019 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 427/850] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:38.588876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7597471-0ec0-48fa-b32a-f52fd71c86ac None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:38.614128 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:38.693618 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd6c8d-2064-4081-88c5-aa7fd462caef None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:38.731219 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:38.758486 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7cbf6-bffd-403a-9caf-531ccea4d19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:38.759561 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7cbf6-bffd-403a-9caf-531ccea4d19b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:38.760021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7cbf6-bffd-403a-9caf-531ccea4d19b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:38.760338 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7cbf6-bffd-403a-9caf-531ccea4d19b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:38.761862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7597471-0ec0-48fa-b32a-f52fd71c86ac None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:38.776037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7cbf6-bffd-403a-9caf-531ccea4d19b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:38.778595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7cbf6-bffd-403a-9caf-531ccea4d19b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:38.805782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b295d9-3583-426b-bf54-d52b22411680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:38.806068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-e4d7cbf6-bffd-403a-9caf-531ccea4d19b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:38.806068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:38.806068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:38.806068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:38.806068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:38.806068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:38.806068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:38.806068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:38.806068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:38.806068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:38.806068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:38.806068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:38.806068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:38.806068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:38.806068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:38.806068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:38.808293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:38.808293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:38.808293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:38.808293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:38.808293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:38.808293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:38.808293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:38.808293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:38.808293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:38.808293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:38.808293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:38.808293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:38.808293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:38.808293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:38.808293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:38.808293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:38.808293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:38.810388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:38.810388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:38.810388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:38.810866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b295d9-3583-426b-bf54-d52b22411680 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:38.810866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b295d9-3583-426b-bf54-d52b22411680 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:38.810866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b295d9-3583-426b-bf54-d52b22411680 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:38.811265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b295d9-3583-426b-bf54-d52b22411680 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:38.811774 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b295d9-3583-426b-bf54-d52b22411680 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:38.813422 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b295d9-3583-426b-bf54-d52b22411680 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:38.816467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 424/851] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:38.816467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7cbf6-bffd-403a-9caf-531ccea4d19b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:38.825988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 428/852] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 16:01:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:38.827424 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b295d9-3583-426b-bf54-d52b22411680 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:39.006606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7cbf6-bffd-403a-9caf-531ccea4d19b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:39.025078 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b295d9-3583-426b-bf54-d52b22411680 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:39.030949 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:39.078726 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e6670-4fb2-4857-a024-775a6f781a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:39.079582 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e6670-4fb2-4857-a024-775a6f781a11 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:39.080000 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e6670-4fb2-4857-a024-775a6f781a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:39.080442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e6670-4fb2-4857-a024-775a6f781a11 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:39.091934 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e6670-4fb2-4857-a024-775a6f781a11 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:39.092200 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe73a7-aa78-4291-9f91-4422c3af94cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:39.093187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe73a7-aa78-4291-9f91-4422c3af94cf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:39.093689 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe73a7-aa78-4291-9f91-4422c3af94cf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:39.094092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e6670-4fb2-4857-a024-775a6f781a11 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:39.094255 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe73a7-aa78-4291-9f91-4422c3af94cf None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:39.095477 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe73a7-aa78-4291-9f91-4422c3af94cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:39.096040 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe73a7-aa78-4291-9f91-4422c3af94cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:39.098955 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe73a7-aa78-4291-9f91-4422c3af94cf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:39.110253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-607e6670-4fb2-4857-a024-775a6f781a11 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:01:39.110253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:39.110253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:39.110253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:39.110253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:39.110253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:39.110253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:39.110253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:39.110253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:39.110253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:39.110253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:39.110253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:39.110253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:39.110253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:39.110253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:39.110253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:39.112408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:39.112408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:39.112408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:39.112408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:39.112408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:39.112408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:39.112408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:39.112408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:39.112408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:39.112408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:39.112408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:39.112408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:39.112408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:39.112408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:39.112408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:39.112408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:39.112408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:39.112408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:39.114794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:39.114794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:39.114794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:39.114794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:39.114794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:39.114794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:39.114794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:39.114794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:39.114794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 16:01:39.114794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:39.118961 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 425/853] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:01:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:39.120841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e6670-4fb2-4857-a024-775a6f781a11 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:39.128380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 429/854] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:39.131039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe73a7-aa78-4291-9f91-4422c3af94cf None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:39.153785 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:39.287315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e6670-4fb2-4857-a024-775a6f781a11 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:39.328161 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f5b57-3fca-45ff-a863-117894bc81fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:39.329126 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f5b57-3fca-45ff-a863-117894bc81fc None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:39.329509 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f5b57-3fca-45ff-a863-117894bc81fc None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:39.329881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f5b57-3fca-45ff-a863-117894bc81fc None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:39.330597 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f5b57-3fca-45ff-a863-117894bc81fc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:39.330995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f5b57-3fca-45ff-a863-117894bc81fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:39.332586 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f5b57-3fca-45ff-a863-117894bc81fc None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:39.346712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 426/855] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 16:01:39 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:39.346712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f5b57-3fca-45ff-a863-117894bc81fc None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:39.347132 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe73a7-aa78-4291-9f91-4422c3af94cf None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:39.391510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:39.416558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d20c2f-218e-4163-b8c4-5c669b816aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:39.417708 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d20c2f-218e-4163-b8c4-5c669b816aff None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:39.418275 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d20c2f-218e-4163-b8c4-5c669b816aff None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:39.418886 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d20c2f-218e-4163-b8c4-5c669b816aff None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:39.420910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d20c2f-218e-4163-b8c4-5c669b816aff None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:39.421650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d20c2f-218e-4163-b8c4-5c669b816aff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:39.424532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d20c2f-218e-4163-b8c4-5c669b816aff None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:39.463379 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 430/856] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Sep 6 16:01:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:39.464599 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d20c2f-218e-4163-b8c4-5c669b816aff None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:39.510512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f5b57-3fca-45ff-a863-117894bc81fc None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:39.545279 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:39.614983 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d20c2f-218e-4163-b8c4-5c669b816aff None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:39.619304 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2a336-e28d-46f6-beac-89aeb1abd96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:39.620522 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2a336-e28d-46f6-beac-89aeb1abd96c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:39.621116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2a336-e28d-46f6-beac-89aeb1abd96c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:39.621671 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2a336-e28d-46f6-beac-89aeb1abd96c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:39.638890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2a336-e28d-46f6-beac-89aeb1abd96c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:39.641277 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2a336-e28d-46f6-beac-89aeb1abd96c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:39.666012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-15c2a336-e28d-46f6-beac-89aeb1abd96c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:39.666012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:39.666012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:39.666012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:39.666012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:39.666012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:39.666012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:39.666012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:39.666012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:39.666012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:39.666012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:39.666012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:39.666012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:39.666012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:39.666012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:39.667009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:39.667009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:39.667009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:39.667009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:39.667009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:39.667009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:39.667009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:39.667009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:39.667009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:39.667009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:39.667009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:39.667009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:39.667009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:39.667009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:39.667009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:39.667009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:39.667009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:39.667940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:39.667940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:39.667940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:39.667940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:39.667940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:39.667940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:39.667940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:39.667940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:39.674938 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 427/857] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:39.676674 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2a336-e28d-46f6-beac-89aeb1abd96c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:39.680290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d0578-e739-43bb-8511-449075632f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:39.681395 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d0578-e739-43bb-8511-449075632f51 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:39.682743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d0578-e739-43bb-8511-449075632f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:39.683190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d0578-e739-43bb-8511-449075632f51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:39.779072 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d0578-e739-43bb-8511-449075632f51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:39.780576 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d0578-e739-43bb-8511-449075632f51 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:39.859725 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:39.872388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2a336-e28d-46f6-beac-89aeb1abd96c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:39.900240 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:39.931242 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea8dc4-a3ea-4924-8d24-e555f7e3258f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:39.932202 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea8dc4-a3ea-4924-8d24-e555f7e3258f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:39.932658 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea8dc4-a3ea-4924-8d24-e555f7e3258f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:39.933113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea8dc4-a3ea-4924-8d24-e555f7e3258f None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:39.934071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea8dc4-a3ea-4924-8d24-e555f7e3258f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:39.934581 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea8dc4-a3ea-4924-8d24-e555f7e3258f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:39.936890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea8dc4-a3ea-4924-8d24-e555f7e3258f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:39.965792 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 428/858] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:39.965792 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb419e0-e9eb-4a25-a894-05f89450a084 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:39.965792 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb419e0-e9eb-4a25-a894-05f89450a084 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:39.965792 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb419e0-e9eb-4a25-a894-05f89450a084 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:39.965792 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 429/859] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:39.967396 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ab44d-fc28-4a6a-be8f-27f66c76c52f None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:39.967775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ab44d-fc28-4a6a-be8f-27f66c76c52f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:39.968154 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ab44d-fc28-4a6a-be8f-27f66c76c52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:39.987151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 431/860] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:01:39 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:01:39.988137 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d0578-e739-43bb-8511-449075632f51 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:40.003024 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9b3ab44d-fc28-4a6a-be8f-27f66c76c52f None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:40.004959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9b3ab44d-fc28-4a6a-be8f-27f66c76c52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:40.119678 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 430/861] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:40.120798 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ab44d-fc28-4a6a-be8f-27f66c76c52f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:40.173494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d0578-e739-43bb-8511-449075632f51 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:40.252439 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ba3a2-3135-4640-8b92-a6c1bbbb8ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:40.253360 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ba3a2-3135-4640-8b92-a6c1bbbb8ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:40.253846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ba3a2-3135-4640-8b92-a6c1bbbb8ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:40.254330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ba3a2-3135-4640-8b92-a6c1bbbb8ab3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:40.256268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ba3a2-3135-4640-8b92-a6c1bbbb8ab3 None admin] RBAC: Adding query filter params (role.id=56d621cf28924933b6356795a6f8ba23, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=ba58a4914ddf418db9e8b9dce5784cf8) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:40.256822 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ba3a2-3135-4640-8b92-a6c1bbbb8ab3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:40.259633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ba3a2-3135-4640-8b92-a6c1bbbb8ab3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:40.276903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 432/862] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:01:39 2022] GET /identity/v3/role_assignments?user.id=ba58a4914ddf418db9e8b9dce5784cf8&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=56d621cf28924933b6356795a6f8ba23 => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:40.330468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:40.352015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ab44d-fc28-4a6a-be8f-27f66c76c52f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:40.419130 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1ebea-e661-446d-a26c-f3e5679f9774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:40.420694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1ebea-e661-446d-a26c-f3e5679f9774 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:40.421485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1ebea-e661-446d-a26c-f3e5679f9774 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:40.422258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1ebea-e661-446d-a26c-f3e5679f9774 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:40.443574 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1ebea-e661-446d-a26c-f3e5679f9774 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:40.446744 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1ebea-e661-446d-a26c-f3e5679f9774 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:40.478480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-56b1ebea-e661-446d-a26c-f3e5679f9774 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:40.478480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:40.478480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:40.478480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:40.478480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:40.478480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:40.478480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:40.478480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:40.478480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:40.478480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:40.478480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:40.478480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:40.478480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:40.478480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:40.478480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:40.478480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:40.480407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:40.480407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:40.480407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:40.480407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:40.480407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:40.480407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:40.480407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:40.480407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:40.480407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:40.480407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:40.480407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:40.480407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:40.480407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:40.480407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:40.480407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:40.480407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:40.480407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:40.482244 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:40.482244 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:40.482244 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:40.485856 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 431/863] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:40.496082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ba3a2-3135-4640-8b92-a6c1bbbb8ab3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:40.512351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1ebea-e661-446d-a26c-f3e5679f9774 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:40.585826 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:40.712753 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ba3a2-3135-4640-8b92-a6c1bbbb8ab3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:40.721212 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1ebea-e661-446d-a26c-f3e5679f9774 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:40.761650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e6a9e-b678-4ec8-9afa-bca32051b549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:40.763454 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e6a9e-b678-4ec8-9afa-bca32051b549 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:40.764001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e6a9e-b678-4ec8-9afa-bca32051b549 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:40.764482 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e6a9e-b678-4ec8-9afa-bca32051b549 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:40.765665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e6a9e-b678-4ec8-9afa-bca32051b549 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:40.766207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e6a9e-b678-4ec8-9afa-bca32051b549 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:40.768178 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e6a9e-b678-4ec8-9afa-bca32051b549 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:40.786015 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 433/864] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:40.789554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6742eb-1508-4b72-8d48-a3348450e178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:40.791155 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6742eb-1508-4b72-8d48-a3348450e178 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:40.791640 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6742eb-1508-4b72-8d48-a3348450e178 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:40.791892 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e6a9e-b678-4ec8-9afa-bca32051b549 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:40.792084 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6742eb-1508-4b72-8d48-a3348450e178 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:40.793086 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6742eb-1508-4b72-8d48-a3348450e178 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:40.795782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6742eb-1508-4b72-8d48-a3348450e178 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:40.801098 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:40.815766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-1e6742eb-1508-4b72-8d48-a3348450e178 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 16:01:40.815766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:40.815766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:01:40.815766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:01:40.815766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:01:40.815766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:01:40.815766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:01:40.815766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:40.815766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 16:01:40.815766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:40.815766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:01:40.815766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:40.815766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:40.815766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:40.815766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:40.815766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:40.816801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:40.816801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:40.816801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:40.816801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:40.816801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:40.816801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:40.816801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:40.816801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:01:40.816801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:01:40.816801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:01:40.816801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:01:40.816801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:40.816801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:40.816801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:40.816801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:40.816801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:40.816801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:40.817914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:40.817914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:40.817914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:40.817914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:40.817914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:40.817914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:40.817914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:40.817914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:40.817914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:40.817914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:40.817914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:01:40.817914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:40.817914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 16:01:40.817914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:40.824940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 432/865] 10.222.0.26 () {56 vars in 1132 bytes} [Tue Sep 6 16:01:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:40.828743 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6742eb-1508-4b72-8d48-a3348450e178 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:40.956334 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:40.971232 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e6a9e-b678-4ec8-9afa-bca32051b549 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:40.987913 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6742eb-1508-4b72-8d48-a3348450e178 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:41.030837 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d72e3e-4417-4eb9-b346-6ee01a85eb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:41.031484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d72e3e-4417-4eb9-b346-6ee01a85eb51 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:41.031823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d72e3e-4417-4eb9-b346-6ee01a85eb51 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:41.032089 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d72e3e-4417-4eb9-b346-6ee01a85eb51 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:41.032735 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d72e3e-4417-4eb9-b346-6ee01a85eb51 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:41.033036 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d72e3e-4417-4eb9-b346-6ee01a85eb51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:41.034425 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d72e3e-4417-4eb9-b346-6ee01a85eb51 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:41.046336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 433/866] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 6 16:01:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:41.047461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d72e3e-4417-4eb9-b346-6ee01a85eb51 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:41.065818 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fca9eb-fc84-4dd1-abd9-5ed564d127ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:41.068157 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fca9eb-fc84-4dd1-abd9-5ed564d127ea None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:41.068733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fca9eb-fc84-4dd1-abd9-5ed564d127ea None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:41.069112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fca9eb-fc84-4dd1-abd9-5ed564d127ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:41.077536 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fca9eb-fc84-4dd1-abd9-5ed564d127ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:41.079155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fca9eb-fc84-4dd1-abd9-5ed564d127ea None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:41.092812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-79fca9eb-fc84-4dd1-abd9-5ed564d127ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:41.092812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:41.092812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:41.092812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:41.092812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:41.092812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:41.092812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:41.092812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:41.092812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:41.092812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:41.092812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:41.092812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:41.092812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:41.092812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:41.092812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:41.094645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:41.094645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:41.094645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:41.094645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:41.094645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:41.094645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:41.094645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:41.094645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:41.094645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:41.094645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:41.094645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:41.094645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:41.094645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:41.094645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:41.094645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:41.094645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:41.094645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:41.097294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:41.097294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:41.097294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:41.097294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:41.097294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:41.097294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:41.097294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:41.097294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:41.097294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 434/867] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:41.100490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fca9eb-fc84-4dd1-abd9-5ed564d127ea None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:41.182591 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:41.232606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d72e3e-4417-4eb9-b346-6ee01a85eb51 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:41.274988 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e976e72-674a-4258-bc91-f3fce55299d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:41.275699 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e976e72-674a-4258-bc91-f3fce55299d5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:41.276041 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e976e72-674a-4258-bc91-f3fce55299d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:41.276374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e976e72-674a-4258-bc91-f3fce55299d5 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:41.277395 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e976e72-674a-4258-bc91-f3fce55299d5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:41.277569 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e976e72-674a-4258-bc91-f3fce55299d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:41.279027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e976e72-674a-4258-bc91-f3fce55299d5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:41.285266 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 434/868] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 6 16:01:41 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:41.297021 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fca9eb-fc84-4dd1-abd9-5ed564d127ea None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:41.356040 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e4f72-9aac-4f9f-b222-b9f5540db2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:41.357053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e4f72-9aac-4f9f-b222-b9f5540db2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:41.357498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e4f72-9aac-4f9f-b222-b9f5540db2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:41.357959 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e4f72-9aac-4f9f-b222-b9f5540db2e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:41.359078 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e4f72-9aac-4f9f-b222-b9f5540db2e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:41.359632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e4f72-9aac-4f9f-b222-b9f5540db2e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:41.361703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e4f72-9aac-4f9f-b222-b9f5540db2e6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:41.382001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 435/869] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:41.388187 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e976e72-674a-4258-bc91-f3fce55299d5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:41.515675 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e976e72-674a-4258-bc91-f3fce55299d5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:41.549750 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:41.568006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e76d76-42d8-4015-96c4-569be5edc13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:41.568664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e76d76-42d8-4015-96c4-569be5edc13c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:41.568950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e76d76-42d8-4015-96c4-569be5edc13c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:41.569216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e76d76-42d8-4015-96c4-569be5edc13c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:41.569979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e76d76-42d8-4015-96c4-569be5edc13c None admin] RBAC: Adding query filter params (role.id=ba05677650e046a48d5f19440d48fbf2, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=ece176d412574790a13d03dcca08f91a) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:41.570289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e76d76-42d8-4015-96c4-569be5edc13c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:41.574708 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e76d76-42d8-4015-96c4-569be5edc13c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:41.581661 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 435/870] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:01:41 2022] GET /identity/v3/role_assignments?user.id=ece176d412574790a13d03dcca08f91a&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:41.664251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80d221-65ad-497f-9c7f-a1ada6a3f117 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:41.664985 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80d221-65ad-497f-9c7f-a1ada6a3f117 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:41.665503 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80d221-65ad-497f-9c7f-a1ada6a3f117 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:41.667581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 436/871] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:41.673990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0c7db-d7ce-496c-9154-352261f5aa27 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:41.674825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0c7db-d7ce-496c-9154-352261f5aa27 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:41.675346 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0c7db-d7ce-496c-9154-352261f5aa27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:41.703790 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-ecb0c7db-d7ce-496c-9154-352261f5aa27 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:41.706503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ecb0c7db-d7ce-496c-9154-352261f5aa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:41.825721 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 436/872] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:42.356329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80d221-65ad-497f-9c7f-a1ada6a3f117 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:42.376540 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:42.560631 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80d221-65ad-497f-9c7f-a1ada6a3f117 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:42.581285 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae781bc3-2431-4c96-bf67-571c8415ef01 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:42.582061 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae781bc3-2431-4c96-bf67-571c8415ef01 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:42.582061 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae781bc3-2431-4c96-bf67-571c8415ef01 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:42.584118 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 437/873] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:01:42.589855 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e9481-7215-4bd2-bf05-386020d781ac None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:42.590128 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e9481-7215-4bd2-bf05-386020d781ac None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:42.590418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e9481-7215-4bd2-bf05-386020d781ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:42.616717 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-ce5e9481-7215-4bd2-bf05-386020d781ac None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:42.617855 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ce5e9481-7215-4bd2-bf05-386020d781ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:42.639892 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bba78a-2f70-4f1f-bcd3-5a6fc3e093a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:42.640513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bba78a-2f70-4f1f-bcd3-5a6fc3e093a4 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:42.640796 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bba78a-2f70-4f1f-bcd3-5a6fc3e093a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:42.641092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bba78a-2f70-4f1f-bcd3-5a6fc3e093a4 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:42.641750 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bba78a-2f70-4f1f-bcd3-5a6fc3e093a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:42.643512 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bba78a-2f70-4f1f-bcd3-5a6fc3e093a4 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:42.670719 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 437/874] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 6 16:01:42 2022] POST /identity/v3/services => generated 251 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:01:42.736895 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 438/875] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:42 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:43.025920 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bba78a-2f70-4f1f-bcd3-5a6fc3e093a4 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:43.111495 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:43.148689 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bba78a-2f70-4f1f-bcd3-5a6fc3e093a4 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:43.188898 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0417c32-3943-40af-b950-3b03227af679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:43.189543 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0417c32-3943-40af-b950-3b03227af679 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:43.189822 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0417c32-3943-40af-b950-3b03227af679 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:43.190104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0417c32-3943-40af-b950-3b03227af679 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:43.209647 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0417c32-3943-40af-b950-3b03227af679 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:01:43.219537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-d0417c32-3943-40af-b950-3b03227af679 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:01:43.219537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:43.219537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:43.219537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:43.219537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:43.219537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:43.219537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:43.219537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:43.219537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:43.219537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:43.219537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:43.219537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:43.219537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:01:43.219537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:01:43.219537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:01:43.219537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:01:43.221421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:01:43.221421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:01:43.221421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:01:43.221421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:01:43.221421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:43.221421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:43.221421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:43.221421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:43.221421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:43.221421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:43.221421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:43.221421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:43.221421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:43.221421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:43.221421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:43.221421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:43.224638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:43.224638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:43.224638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:43.224638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:43.224638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:01:43.224638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:01:43.224638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:01:43.224638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:01:43.224638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:01:43.224638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:01:43.224638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:01:43.224638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:43.224638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 438/876] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Sep 6 16:01:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:43.232687 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e9481-7215-4bd2-bf05-386020d781ac None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:43.334941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:43.436446 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e9481-7215-4bd2-bf05-386020d781ac None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:43.506051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9593aa2-59e4-4e99-804b-be739af194b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:43.507154 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9593aa2-59e4-4e99-804b-be739af194b8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:43.507596 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9593aa2-59e4-4e99-804b-be739af194b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:43.508027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9593aa2-59e4-4e99-804b-be739af194b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:43.509196 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9593aa2-59e4-4e99-804b-be739af194b8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:43.509670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9593aa2-59e4-4e99-804b-be739af194b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:43.512165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9593aa2-59e4-4e99-804b-be739af194b8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:43.529709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 439/877] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Sep 6 16:01:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:43.539787 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0417c32-3943-40af-b950-3b03227af679 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:43.736331 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0417c32-3943-40af-b950-3b03227af679 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:43.799339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:43.811409 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574deba-3492-42b6-95c8-796b1bfeae33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:43.813354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574deba-3492-42b6-95c8-796b1bfeae33 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:43.814479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574deba-3492-42b6-95c8-796b1bfeae33 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:43.815577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574deba-3492-42b6-95c8-796b1bfeae33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:43.840952 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574deba-3492-42b6-95c8-796b1bfeae33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:43.843742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574deba-3492-42b6-95c8-796b1bfeae33 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:43.874479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-f574deba-3492-42b6-95c8-796b1bfeae33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:43.874479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:43.874479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:43.874479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:43.874479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:43.874479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:43.874479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:43.874479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:43.874479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:43.874479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:43.874479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:43.874479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:43.874479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:43.874479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:43.874479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:43.874479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:43.876269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:43.876269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:43.876269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:43.876269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:43.876269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:43.876269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:43.876269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:43.876269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:43.876269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:43.876269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:43.876269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:43.876269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:43.876269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:43.876269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:43.876269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:43.876269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:43.876269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:43.877980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:43.877980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:43.877980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:43.877980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 439/878] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:43.887862 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9593aa2-59e4-4e99-804b-be739af194b8 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:44.007113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9593aa2-59e4-4e99-804b-be739af194b8 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:44.009954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f51a2-8269-4e32-b08c-b335737d99b6 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:44.010640 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f51a2-8269-4e32-b08c-b335737d99b6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:44.011189 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f51a2-8269-4e32-b08c-b335737d99b6 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:44.014776 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 440/879] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:01:44.019280 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:44.023327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a94ac-6d9a-4768-8867-d02070f0f239 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:44.023791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a94ac-6d9a-4768-8867-d02070f0f239 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:44.024266 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a94ac-6d9a-4768-8867-d02070f0f239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:44.057068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7cbfe-39e1-46db-b44a-055fc5ce4288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:44.058013 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7cbfe-39e1-46db-b44a-055fc5ce4288 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:44.058477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7cbfe-39e1-46db-b44a-055fc5ce4288 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:44.059011 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7cbfe-39e1-46db-b44a-055fc5ce4288 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:44.060233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7cbfe-39e1-46db-b44a-055fc5ce4288 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:44.060754 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7cbfe-39e1-46db-b44a-055fc5ce4288 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:44.061738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8c7a94ac-6d9a-4768-8867-d02070f0f239 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:44.063179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7cbfe-39e1-46db-b44a-055fc5ce4288 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:44.064253 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8c7a94ac-6d9a-4768-8867-d02070f0f239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:44.086724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 440/880] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:44.093051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7cbfe-39e1-46db-b44a-055fc5ce4288 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:44.201209 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 441/881] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:44 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:01:44.238881 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cbb21-2000-427d-bfe3-465c094f1ecd None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:44.239455 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cbb21-2000-427d-bfe3-465c094f1ecd None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:44.239932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cbb21-2000-427d-bfe3-465c094f1ecd None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:44.249353 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 442/882] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:44 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:01:44.251730 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed6f1a-5c28-42fb-950b-256bb209082e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:44.252230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed6f1a-5c28-42fb-950b-256bb209082e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:44.252723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed6f1a-5c28-42fb-950b-256bb209082e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:44.285459 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c0ed6f1a-5c28-42fb-950b-256bb209082e None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:44.287469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c0ed6f1a-5c28-42fb-950b-256bb209082e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:44.320291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7cbfe-39e1-46db-b44a-055fc5ce4288 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:44.402399 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87491cf4-d158-4208-989a-1453470cf5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:44.403951 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87491cf4-d158-4208-989a-1453470cf5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:44.404526 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87491cf4-d158-4208-989a-1453470cf5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:44.405087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87491cf4-d158-4208-989a-1453470cf5d8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:44.424250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87491cf4-d158-4208-989a-1453470cf5d8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:44.427882 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87491cf4-d158-4208-989a-1453470cf5d8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:44.434895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 443/883] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:44 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:01:44.456826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-87491cf4-d158-4208-989a-1453470cf5d8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:01:44.456826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:44.456826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:44.456826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:44.456826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:44.456826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:44.456826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:44.456826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:44.456826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:44.456826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:44.456826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:44.456826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:44.456826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:44.456826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:44.456826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:44.456826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:44.459208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:44.459208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:44.459208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:44.459208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:44.459208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:44.459208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:44.459208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:44.459208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:44.459208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:44.459208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:44.459208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:44.459208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:44.459208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:44.459208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:44.459208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:44.459208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:44.459208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:44.459208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:44.462941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:44.462941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:44.462941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:44.462941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:44.462941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:44.462941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:44.462941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:44.462941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:44.462941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:01:44.462941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:44.462941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 441/884] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 6 16:01:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:44.470714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed6f1a-5c28-42fb-950b-256bb209082e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:44.486137 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87491cf4-d158-4208-989a-1453470cf5d8 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:44.531469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:44.579290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:44.632047 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87491cf4-d158-4208-989a-1453470cf5d8 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:44.678264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794030a-294e-45e3-b6df-29cd45e62bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:44.679539 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794030a-294e-45e3-b6df-29cd45e62bca None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:44.680030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794030a-294e-45e3-b6df-29cd45e62bca None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:44.680497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794030a-294e-45e3-b6df-29cd45e62bca None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:44.681936 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794030a-294e-45e3-b6df-29cd45e62bca None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:44.683694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794030a-294e-45e3-b6df-29cd45e62bca None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:44.700078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-7794030a-294e-45e3-b6df-29cd45e62bca None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:01:44.700078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:44.700078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:01:44.700078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:01:44.700078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:01:44.700078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:01:44.700078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:01:44.700078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:44.700078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:01:44.700078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:44.700078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:01:44.700078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:44.700078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:44.700078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:44.700078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:44.700078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:44.700078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:44.701131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:44.701131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:44.701131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:44.701131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:44.701131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:44.701131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:44.701131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:01:44.701131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:01:44.701131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:01:44.701131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:01:44.701131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:44.701131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:44.701131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:44.701131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:44.701131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:44.701131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:44.701967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:44.701967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:44.701967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:44.701967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:44.701967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:44.701967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:44.701967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:44.701967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:44.701967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:44.701967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:44.701967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:01:44.701967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:44.701967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:01:44.701967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:44.709726 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 442/885] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 6 16:01:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:44.712380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794030a-294e-45e3-b6df-29cd45e62bca None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:44.719981 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed6f1a-5c28-42fb-950b-256bb209082e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:44.785013 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611955ea-26c4-4f38-bba4-14bb5923d276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:44.786300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611955ea-26c4-4f38-bba4-14bb5923d276 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:44.786945 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611955ea-26c4-4f38-bba4-14bb5923d276 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:44.787421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611955ea-26c4-4f38-bba4-14bb5923d276 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:44.788769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611955ea-26c4-4f38-bba4-14bb5923d276 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:44.789266 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611955ea-26c4-4f38-bba4-14bb5923d276 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:44.791884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611955ea-26c4-4f38-bba4-14bb5923d276 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:44.831255 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 444/886] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Sep 6 16:01:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:44.836360 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611955ea-26c4-4f38-bba4-14bb5923d276 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:44.839850 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:44.850234 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794030a-294e-45e3-b6df-29cd45e62bca None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:44.896885 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a9dd2-4734-4ed8-bca4-30217d2dc96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:44.898617 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a9dd2-4734-4ed8-bca4-30217d2dc96c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:44.899139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a9dd2-4734-4ed8-bca4-30217d2dc96c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:44.899618 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a9dd2-4734-4ed8-bca4-30217d2dc96c None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:44.900751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a9dd2-4734-4ed8-bca4-30217d2dc96c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:44.901267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a9dd2-4734-4ed8-bca4-30217d2dc96c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:44.903903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a9dd2-4734-4ed8-bca4-30217d2dc96c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:44.916186 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 443/887] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 6 16:01:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:44.917909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a9dd2-4734-4ed8-bca4-30217d2dc96c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:45.025569 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:45.029559 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611955ea-26c4-4f38-bba4-14bb5923d276 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:45.069947 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a9dd2-4734-4ed8-bca4-30217d2dc96c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:45.076588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d39239-52b6-4597-8aeb-e43cdf93c699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:45.077209 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d39239-52b6-4597-8aeb-e43cdf93c699 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:45.077478 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d39239-52b6-4597-8aeb-e43cdf93c699 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:45.077754 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d39239-52b6-4597-8aeb-e43cdf93c699 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:45.137690 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42b068-0ad3-4cb6-810d-c1d493467365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:45.138742 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42b068-0ad3-4cb6-810d-c1d493467365 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:45.139230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42b068-0ad3-4cb6-810d-c1d493467365 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:45.139719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42b068-0ad3-4cb6-810d-c1d493467365 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:45.140887 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42b068-0ad3-4cb6-810d-c1d493467365 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:45.141421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42b068-0ad3-4cb6-810d-c1d493467365 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:45.144073 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42b068-0ad3-4cb6-810d-c1d493467365 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:45.151219 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 444/888] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 6 16:01:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:45.152056 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42b068-0ad3-4cb6-810d-c1d493467365 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:45.191733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:45.192382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d39239-52b6-4597-8aeb-e43cdf93c699 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:45.197227 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d39239-52b6-4597-8aeb-e43cdf93c699 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:45.337925 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42b068-0ad3-4cb6-810d-c1d493467365 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:45.374425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:45.382939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 445/889] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:01:44 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:01:45.391617 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d39239-52b6-4597-8aeb-e43cdf93c699 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:45.417595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483ad9c-94d2-4246-8e39-93f32d2aaa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:45.418648 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483ad9c-94d2-4246-8e39-93f32d2aaa34 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:45.419191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483ad9c-94d2-4246-8e39-93f32d2aaa34 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:45.419678 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483ad9c-94d2-4246-8e39-93f32d2aaa34 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:45.420939 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483ad9c-94d2-4246-8e39-93f32d2aaa34 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:45.423661 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483ad9c-94d2-4246-8e39-93f32d2aaa34 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:45.458589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 445/890] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 6 16:01:45 2022] POST /identity/v3/services => generated 269 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:01:45.537030 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d39239-52b6-4597-8aeb-e43cdf93c699 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:45.548316 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e9277-0fef-4c5d-a543-f6c61f2775b6 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:45.548776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e9277-0fef-4c5d-a543-f6c61f2775b6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:45.549227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e9277-0fef-4c5d-a543-f6c61f2775b6 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:45.552847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 446/891] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:01:45.561523 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6dce2-e17c-465c-bd23-952836a5b3c3 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:45.561973 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6dce2-e17c-465c-bd23-952836a5b3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:45.562558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6dce2-e17c-465c-bd23-952836a5b3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:45.608607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f8d6dce2-e17c-465c-bd23-952836a5b3c3 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:45.610561 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f8d6dce2-e17c-465c-bd23-952836a5b3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:45.618265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4cc1c-7715-407c-8fd3-8c723a002e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:45.619443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4cc1c-7715-407c-8fd3-8c723a002e33 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:45.619905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4cc1c-7715-407c-8fd3-8c723a002e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:45.620357 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4cc1c-7715-407c-8fd3-8c723a002e33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:45.643586 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d4cc1c-7715-407c-8fd3-8c723a002e33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:45.647643 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d4cc1c-7715-407c-8fd3-8c723a002e33 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:45.679203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-a0d4cc1c-7715-407c-8fd3-8c723a002e33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:45.679203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:45.679203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:45.679203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:45.679203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:45.679203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:45.679203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:45.679203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:45.679203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:45.679203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:45.679203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:45.679203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:45.679203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:45.679203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:45.679203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:45.679203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:45.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:45.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:45.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:45.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:45.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:45.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:45.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:45.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:45.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:45.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:45.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:45.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:45.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:45.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:45.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:45.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:45.680296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:45.681252 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:45.681252 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:45.681252 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:45.681439 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 446/892] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:45.691220 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4cc1c-7715-407c-8fd3-8c723a002e33 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:45.746355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 447/893] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:01:45.789148 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:45.885768 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4cc1c-7715-407c-8fd3-8c723a002e33 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:45.952500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10aaf9-3a19-4338-b8e9-1f35ee4cee99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:45.953507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10aaf9-3a19-4338-b8e9-1f35ee4cee99 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:45.953996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10aaf9-3a19-4338-b8e9-1f35ee4cee99 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:45.954476 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10aaf9-3a19-4338-b8e9-1f35ee4cee99 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:45.955694 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10aaf9-3a19-4338-b8e9-1f35ee4cee99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:45.956236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10aaf9-3a19-4338-b8e9-1f35ee4cee99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:45.959509 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10aaf9-3a19-4338-b8e9-1f35ee4cee99 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:45.985367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 447/894] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:45.995840 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6dce2-e17c-465c-bd23-952836a5b3c3 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:46.012270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:46.127112 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6dce2-e17c-465c-bd23-952836a5b3c3 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:46.167401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed7a20-af79-4c12-bb12-cbeeb5be84d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:46.168407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed7a20-af79-4c12-bb12-cbeeb5be84d4 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:46.168922 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed7a20-af79-4c12-bb12-cbeeb5be84d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:46.169412 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed7a20-af79-4c12-bb12-cbeeb5be84d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:46.178091 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed7a20-af79-4c12-bb12-cbeeb5be84d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:46.180054 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed7a20-af79-4c12-bb12-cbeeb5be84d4 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:46.194119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-2fed7a20-af79-4c12-bb12-cbeeb5be84d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:46.194119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:46.194119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:46.194119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:46.194119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:46.194119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:46.194119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:46.194119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:46.194119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:46.194119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:46.194119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:46.194119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:46.194119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:46.194119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:46.194119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:46.195160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:46.195160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:46.195160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:46.195160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:46.195160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:46.195160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:46.195160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:46.195160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:46.195160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:46.195160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:46.195160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:46.195160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:46.195160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:46.195160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:46.195160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:46.195160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:46.195160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:46.196032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:46.196032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:46.196032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:46.196032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:46.196032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:46.196032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:46.196032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:46.196032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:46.198717 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 448/895] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:46.207531 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10aaf9-3a19-4338-b8e9-1f35ee4cee99 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:46.268420 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed7a20-af79-4c12-bb12-cbeeb5be84d4 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:46.374306 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10aaf9-3a19-4338-b8e9-1f35ee4cee99 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:46.387059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:46.420669 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f957dd-d8a3-4587-a721-798df8c79b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:46.421049 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f957dd-d8a3-4587-a721-798df8c79b74 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:46.421331 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f957dd-d8a3-4587-a721-798df8c79b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:46.421603 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f957dd-d8a3-4587-a721-798df8c79b74 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:46.422313 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f957dd-d8a3-4587-a721-798df8c79b74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:46.422633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f957dd-d8a3-4587-a721-798df8c79b74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:46.424143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f957dd-d8a3-4587-a721-798df8c79b74 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:46.446645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 448/896] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:46.447742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f957dd-d8a3-4587-a721-798df8c79b74 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:46.451075 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:46.531394 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed7a20-af79-4c12-bb12-cbeeb5be84d4 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:46.599974 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab4e60-703b-4ac9-afe5-e8c63d81c478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:46.601053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab4e60-703b-4ac9-afe5-e8c63d81c478 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:46.601540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab4e60-703b-4ac9-afe5-e8c63d81c478 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:46.602033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab4e60-703b-4ac9-afe5-e8c63d81c478 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:46.602976 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab4e60-703b-4ac9-afe5-e8c63d81c478 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:46.604626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab4e60-703b-4ac9-afe5-e8c63d81c478 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:46.615733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-c3ab4e60-703b-4ac9-afe5-e8c63d81c478 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 16:01:46.615733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:46.615733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:01:46.615733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:01:46.615733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:01:46.615733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:01:46.615733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:01:46.615733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:46.615733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 16:01:46.615733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:46.615733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:01:46.615733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:46.615733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:46.615733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:46.615733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:46.615733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:46.615733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:46.616867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:46.616867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:46.616867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:46.616867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:46.616867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:46.616867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:46.616867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:01:46.616867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:01:46.616867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:01:46.616867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:01:46.616867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:46.616867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:46.616867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:46.616867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:46.616867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:46.616867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:46.617728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:46.617728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:46.617728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:46.617728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:46.617728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:46.617728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:46.617728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:46.617728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:46.617728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:46.617728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:46.617728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:01:46.617728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:46.617728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 16:01:46.617728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:46.620592 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 449/897] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 16:01:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:46.632643 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab4e60-703b-4ac9-afe5-e8c63d81c478 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:46.637047 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f957dd-d8a3-4587-a721-798df8c79b74 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:46.709505 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68bce2-1c01-4236-92c8-8fa984bd9271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:46.711031 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68bce2-1c01-4236-92c8-8fa984bd9271 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:46.711815 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68bce2-1c01-4236-92c8-8fa984bd9271 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:46.712593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68bce2-1c01-4236-92c8-8fa984bd9271 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:46.714273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68bce2-1c01-4236-92c8-8fa984bd9271 None admin] RBAC: Adding query filter params (role.id=91f753ea35ce4f5a953d0a503481da1a, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=ba58a4914ddf418db9e8b9dce5784cf8) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:46.715310 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68bce2-1c01-4236-92c8-8fa984bd9271 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:46.718611 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68bce2-1c01-4236-92c8-8fa984bd9271 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:46.735104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 449/898] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:01:46 2022] GET /identity/v3/role_assignments?user.id=ba58a4914ddf418db9e8b9dce5784cf8&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=91f753ea35ce4f5a953d0a503481da1a => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:46.856705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab4e60-703b-4ac9-afe5-e8c63d81c478 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:46.931762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ea174-ee3f-416d-afb8-ea6e865cc252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:46.932812 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ea174-ee3f-416d-afb8-ea6e865cc252 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:46.933294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ea174-ee3f-416d-afb8-ea6e865cc252 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:46.933782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ea174-ee3f-416d-afb8-ea6e865cc252 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:46.935042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ea174-ee3f-416d-afb8-ea6e865cc252 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:46.935589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ea174-ee3f-416d-afb8-ea6e865cc252 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:46.938268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ea174-ee3f-416d-afb8-ea6e865cc252 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:46.942908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:46.950783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 450/899] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 16:01:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:46.964760 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68bce2-1c01-4236-92c8-8fa984bd9271 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:47.020068 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:47.154041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68bce2-1c01-4236-92c8-8fa984bd9271 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:47.219880 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f56ff-a2bf-4af2-a161-27c28165c624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:47.221269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f56ff-a2bf-4af2-a161-27c28165c624 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:47.221988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f56ff-a2bf-4af2-a161-27c28165c624 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:47.222743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f56ff-a2bf-4af2-a161-27c28165c624 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:47.224765 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f56ff-a2bf-4af2-a161-27c28165c624 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:47.225551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f56ff-a2bf-4af2-a161-27c28165c624 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:47.228357 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f56ff-a2bf-4af2-a161-27c28165c624 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:47.238555 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 450/900] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 16:01:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:47.248830 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ea174-ee3f-416d-afb8-ea6e865cc252 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:47.412524 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ea174-ee3f-416d-afb8-ea6e865cc252 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:47.451074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1de50-3dc5-468d-81ef-0f9f73e4f5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:47.451696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1de50-3dc5-468d-81ef-0f9f73e4f5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:47.451975 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1de50-3dc5-468d-81ef-0f9f73e4f5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:47.452248 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1de50-3dc5-468d-81ef-0f9f73e4f5b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:47.452952 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1de50-3dc5-468d-81ef-0f9f73e4f5b1 None admin] RBAC: Adding query filter params (interface=public, service_id=32ace831d45a453298473c60e04171f2, region_id=RegionOne) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:47.453265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1de50-3dc5-468d-81ef-0f9f73e4f5b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:47.454687 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1de50-3dc5-468d-81ef-0f9f73e4f5b1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:47.461433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 451/901] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Sep 6 16:01:47 2022] GET /identity/v3/endpoints?service_id=32ace831d45a453298473c60e04171f2&interface=public®ion_id=RegionOne => generated 194 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:47.470223 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f56ff-a2bf-4af2-a161-27c28165c624 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:47.660520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:47.673306 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f56ff-a2bf-4af2-a161-27c28165c624 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:47.715488 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7e479-2b6d-43b7-9171-1c22bc2a91e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:47.716106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7e479-2b6d-43b7-9171-1c22bc2a91e5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:47.716395 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7e479-2b6d-43b7-9171-1c22bc2a91e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:47.716655 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7e479-2b6d-43b7-9171-1c22bc2a91e5 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:47.717266 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7e479-2b6d-43b7-9171-1c22bc2a91e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:47.718647 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7e479-2b6d-43b7-9171-1c22bc2a91e5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:47.726523 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 451/902] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Sep 6 16:01:47 2022] GET /identity/v3/services => generated 1268 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:01:47.915754 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb32b59-f44e-4de4-a60c-c9f1d510afe0 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:47.916056 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb32b59-f44e-4de4-a60c-c9f1d510afe0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:47.916390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb32b59-f44e-4de4-a60c-c9f1d510afe0 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:47.917963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 452/903] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:01:47.924272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6f73e-d6e0-446a-b831-4255e1e08f05 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:47.924734 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6f73e-d6e0-446a-b831-4255e1e08f05 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:47.925152 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6f73e-d6e0-446a-b831-4255e1e08f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:47.948364 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-dba6f73e-d6e0-446a-b831-4255e1e08f05 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:47.949923 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-dba6f73e-d6e0-446a-b831-4255e1e08f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:48.030062 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 452/904] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:01:48.307352 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63adb23d-0866-4d34-8339-506aab2ec1d8 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:48.308334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63adb23d-0866-4d34-8339-506aab2ec1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:48.308822 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63adb23d-0866-4d34-8339-506aab2ec1d8 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:48.310918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 453/905] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:48.316401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d04ca7-7135-4c55-a19f-045aa9938b67 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:48.316711 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d04ca7-7135-4c55-a19f-045aa9938b67 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:48.316969 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d04ca7-7135-4c55-a19f-045aa9938b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:48.319324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63adb23d-0866-4d34-8339-506aab2ec1d8 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:48.340657 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d2d04ca7-7135-4c55-a19f-045aa9938b67 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:48.342407 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d2d04ca7-7135-4c55-a19f-045aa9938b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:48.409269 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:48.434757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 453/906] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:48.477319 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63adb23d-0866-4d34-8339-506aab2ec1d8 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:48.552498 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a09109-e390-4430-96f4-0a41b61fe6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:48.553159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a09109-e390-4430-96f4-0a41b61fe6de None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:48.553444 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a09109-e390-4430-96f4-0a41b61fe6de None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:48.553714 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a09109-e390-4430-96f4-0a41b61fe6de None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:48.554310 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a09109-e390-4430-96f4-0a41b61fe6de None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:48.556018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a09109-e390-4430-96f4-0a41b61fe6de None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:48.572294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-e1a09109-e390-4430-96f4-0a41b61fe6de None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 16:01:48.572294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:48.572294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:01:48.572294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:01:48.572294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:01:48.572294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:01:48.572294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:01:48.572294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:48.572294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 16:01:48.572294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:48.572294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:01:48.572294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:48.572294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:48.572294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:48.572294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:48.572294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:48.573351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:48.573351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:48.573351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:48.573351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:48.573351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:48.573351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:48.573351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:48.573351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:01:48.573351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:01:48.573351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:01:48.573351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:01:48.573351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:48.573351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:48.573351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:48.573351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:48.573351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:48.573351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:48.574264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:48.574264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:48.574264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:48.574264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:48.574264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:48.574264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:48.574264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:48.574264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:48.574264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:48.574264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:48.574264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:01:48.574264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:48.574264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 16:01:48.574264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:48.575111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 454/907] 10.222.0.26 () {56 vars in 1132 bytes} [Tue Sep 6 16:01:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:48.585679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d04ca7-7135-4c55-a19f-045aa9938b67 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:48.611361 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:48.736634 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d04ca7-7135-4c55-a19f-045aa9938b67 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:48.802808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dd7ce-f1c4-4541-becf-5f0aa3a68619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:48.803838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dd7ce-f1c4-4541-becf-5f0aa3a68619 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:48.804321 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dd7ce-f1c4-4541-becf-5f0aa3a68619 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:48.804814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dd7ce-f1c4-4541-becf-5f0aa3a68619 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:48.805948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22dd7ce-f1c4-4541-becf-5f0aa3a68619 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:48.806511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22dd7ce-f1c4-4541-becf-5f0aa3a68619 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:48.809116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22dd7ce-f1c4-4541-becf-5f0aa3a68619 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:48.819077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 454/908] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 6 16:01:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:48.825012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a09109-e390-4430-96f4-0a41b61fe6de None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:48.916154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dd7ce-f1c4-4541-becf-5f0aa3a68619 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:49.011098 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a09109-e390-4430-96f4-0a41b61fe6de None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:49.067893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:49.078256 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:49.087204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358d072-2612-42e2-93f7-6ec0072fb8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:49.088997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358d072-2612-42e2-93f7-6ec0072fb8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:49.089875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358d072-2612-42e2-93f7-6ec0072fb8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:49.090648 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358d072-2612-42e2-93f7-6ec0072fb8b2 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:49.092186 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358d072-2612-42e2-93f7-6ec0072fb8b2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:49.093085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358d072-2612-42e2-93f7-6ec0072fb8b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:49.096092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358d072-2612-42e2-93f7-6ec0072fb8b2 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:49.107094 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 455/909] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 6 16:01:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:49.114324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358d072-2612-42e2-93f7-6ec0072fb8b2 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:49.118290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dd7ce-f1c4-4541-becf-5f0aa3a68619 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:49.162777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39937d70-b791-4110-8dc5-cb522dcddf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:49.163791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39937d70-b791-4110-8dc5-cb522dcddf62 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:49.164320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39937d70-b791-4110-8dc5-cb522dcddf62 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:49.164875 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39937d70-b791-4110-8dc5-cb522dcddf62 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:49.165942 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39937d70-b791-4110-8dc5-cb522dcddf62 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:49.167749 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39937d70-b791-4110-8dc5-cb522dcddf62 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:49.189758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 455/910] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 6 16:01:48 2022] POST /identity/v3/services => generated 241 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:01:49.323382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358d072-2612-42e2-93f7-6ec0072fb8b2 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:49.401053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2063ff-f3dc-400c-93a8-eb4d02a0b037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:49.402065 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2063ff-f3dc-400c-93a8-eb4d02a0b037 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:49.402501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2063ff-f3dc-400c-93a8-eb4d02a0b037 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:49.403023 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2063ff-f3dc-400c-93a8-eb4d02a0b037 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:49.404171 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2063ff-f3dc-400c-93a8-eb4d02a0b037 None admin] RBAC: Adding query filter params (interface=public, service_id=88f3f0c2ca4949839cfa422a9dff15e4, region_id=RegionOne) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:49.404617 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2063ff-f3dc-400c-93a8-eb4d02a0b037 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:49.406824 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2063ff-f3dc-400c-93a8-eb4d02a0b037 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:49.417798 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 456/911] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Sep 6 16:01:49 2022] GET /identity/v3/endpoints?service_id=88f3f0c2ca4949839cfa422a9dff15e4&interface=public®ion_id=RegionOne => generated 194 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:49.423997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39937d70-b791-4110-8dc5-cb522dcddf62 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:49.618290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39937d70-b791-4110-8dc5-cb522dcddf62 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:49.682773 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0bc33-2f16-4393-b671-e221ef58e63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:49.683728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0bc33-2f16-4393-b671-e221ef58e63f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:49.684229 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0bc33-2f16-4393-b671-e221ef58e63f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:49.684729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0bc33-2f16-4393-b671-e221ef58e63f None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:49.685863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0bc33-2f16-4393-b671-e221ef58e63f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:49.688310 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0bc33-2f16-4393-b671-e221ef58e63f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:49.700815 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 456/912] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Sep 6 16:01:49 2022] GET /identity/v3/services => generated 1497 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:01:49.770698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047607d6-15d1-4fca-860a-2e1fc1065fe6 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:49.771353 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047607d6-15d1-4fca-860a-2e1fc1065fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:49.771913 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047607d6-15d1-4fca-860a-2e1fc1065fe6 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:49.773881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 457/913] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:49.780010 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d82ed-bc7c-4ba0-a282-8691fd7141e6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:49.781091 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d82ed-bc7c-4ba0-a282-8691fd7141e6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:49.781566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d82ed-bc7c-4ba0-a282-8691fd7141e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:49.803826 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-9c3d82ed-bc7c-4ba0-a282-8691fd7141e6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:49.805070 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9c3d82ed-bc7c-4ba0-a282-8691fd7141e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:49.807823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:49.922144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 457/914] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:50.205228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047607d6-15d1-4fca-860a-2e1fc1065fe6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:50.220026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac02ea1-5273-45ad-803c-e0cb3d8444c7 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:50.220316 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac02ea1-5273-45ad-803c-e0cb3d8444c7 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:50.220651 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac02ea1-5273-45ad-803c-e0cb3d8444c7 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:50.224505 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 458/915] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:50.230914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cf990-7e73-421f-b8fb-f07f6230ba22 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:50.231162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cf990-7e73-421f-b8fb-f07f6230ba22 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:50.231333 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cf990-7e73-421f-b8fb-f07f6230ba22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:50.263170 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-738cf990-7e73-421f-b8fb-f07f6230ba22 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:50.266113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-738cf990-7e73-421f-b8fb-f07f6230ba22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:50.287315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:50.344792 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047607d6-15d1-4fca-860a-2e1fc1065fe6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:50.383237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b400731d-fe8c-4a61-8cbc-d3f4c53e72f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:50.383891 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400731d-fe8c-4a61-8cbc-d3f4c53e72f4 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:50.384178 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400731d-fe8c-4a61-8cbc-d3f4c53e72f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:50.384444 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400731d-fe8c-4a61-8cbc-d3f4c53e72f4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:50.397749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b400731d-fe8c-4a61-8cbc-d3f4c53e72f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:01:50.402834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-b400731d-fe8c-4a61-8cbc-d3f4c53e72f4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:01:50.402834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:50.402834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:50.402834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:50.402834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:50.402834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:50.402834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:50.402834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:50.402834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:50.402834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:50.402834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:50.402834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:50.402834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:01:50.402834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:01:50.402834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:01:50.402834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:01:50.403811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:01:50.403811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:01:50.403811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:01:50.403811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:01:50.403811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:50.403811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:50.403811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:50.403811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:50.403811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:50.403811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:50.403811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:50.403811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:50.403811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:50.403811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:50.403811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:50.403811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:50.404673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:50.404673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:50.404673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:50.404673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:50.404673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:01:50.404673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:01:50.404673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:01:50.404673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:01:50.404673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:01:50.404673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:01:50.404673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:01:50.404673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:50.404673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 458/916] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Sep 6 16:01:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:50.411522 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b400731d-fe8c-4a61-8cbc-d3f4c53e72f4 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:50.425296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 459/917] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:01:50.528417 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b400731d-fe8c-4a61-8cbc-d3f4c53e72f4 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:50.543875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:50.570694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7a185-2565-451c-8183-a4655fd5c3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:50.570694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7a185-2565-451c-8183-a4655fd5c3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:50.570694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7a185-2565-451c-8183-a4655fd5c3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:50.570694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7a185-2565-451c-8183-a4655fd5c3a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:50.571357 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c7a185-2565-451c-8183-a4655fd5c3a8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:50.571357 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c7a185-2565-451c-8183-a4655fd5c3a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:50.571853 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c7a185-2565-451c-8183-a4655fd5c3a8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:50.581561 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 459/918] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Sep 6 16:01:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:50.587849 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cf990-7e73-421f-b8fb-f07f6230ba22 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:50.683125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:50.721316 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cf990-7e73-421f-b8fb-f07f6230ba22 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:50.763434 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9e9ff-a33e-45e8-94e9-03c32e3cc78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:50.764088 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9e9ff-a33e-45e8-94e9-03c32e3cc78a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:50.764379 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9e9ff-a33e-45e8-94e9-03c32e3cc78a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:50.764625 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9e9ff-a33e-45e8-94e9-03c32e3cc78a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:50.777060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9e9ff-a33e-45e8-94e9-03c32e3cc78a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:50.778732 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9e9ff-a33e-45e8-94e9-03c32e3cc78a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:50.796154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-0fb9e9ff-a33e-45e8-94e9-03c32e3cc78a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:50.796154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:50.796154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:50.796154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:50.796154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:50.796154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:50.796154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:50.796154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:50.796154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:50.796154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:50.796154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:50.796154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:50.796154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:50.796154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:50.796154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:50.796154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:50.798641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:50.798641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:50.798641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:50.798641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:50.798641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:50.798641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:50.798641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:50.798641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:50.798641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:50.798641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:50.798641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:50.798641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:50.798641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:50.798641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:50.798641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:50.798641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:50.798641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:50.800382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:50.800382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:50.800382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:50.800382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 460/919] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:50.804745 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7a185-2565-451c-8183-a4655fd5c3a8 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:50.930254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7a185-2565-451c-8183-a4655fd5c3a8 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:50.959975 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9e9ff-a33e-45e8-94e9-03c32e3cc78a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:50.978042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e78272-26f5-4ae3-bedf-38e1a1f26086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:50.979066 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e78272-26f5-4ae3-bedf-38e1a1f26086 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:50.979514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e78272-26f5-4ae3-bedf-38e1a1f26086 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:50.979879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e78272-26f5-4ae3-bedf-38e1a1f26086 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:50.980877 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e78272-26f5-4ae3-bedf-38e1a1f26086 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:50.981338 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e78272-26f5-4ae3-bedf-38e1a1f26086 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:50.983860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e78272-26f5-4ae3-bedf-38e1a1f26086 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:51.005252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 460/920] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:51.012627 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e78272-26f5-4ae3-bedf-38e1a1f26086 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:51.057937 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:51.139749 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9e9ff-a33e-45e8-94e9-03c32e3cc78a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:51.147753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e78272-26f5-4ae3-bedf-38e1a1f26086 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:51.178628 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:51.191575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccd8b5-9c50-4c0b-888d-08640380f536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:51.192437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccd8b5-9c50-4c0b-888d-08640380f536 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:51.192820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccd8b5-9c50-4c0b-888d-08640380f536 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:51.193278 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccd8b5-9c50-4c0b-888d-08640380f536 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:51.203049 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccd8b5-9c50-4c0b-888d-08640380f536 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:51.204608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccd8b5-9c50-4c0b-888d-08640380f536 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:51.206860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe76d56-4ff2-4cc6-b5aa-a590f78d6889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:51.209340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe76d56-4ff2-4cc6-b5aa-a590f78d6889 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:51.210135 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe76d56-4ff2-4cc6-b5aa-a590f78d6889 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:51.211089 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe76d56-4ff2-4cc6-b5aa-a590f78d6889 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:51.212375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe76d56-4ff2-4cc6-b5aa-a590f78d6889 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:51.215186 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe76d56-4ff2-4cc6-b5aa-a590f78d6889 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:51.218916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-55ccd8b5-9c50-4c0b-888d-08640380f536 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:01:51.218916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:51.218916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:51.218916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:51.218916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:51.218916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:51.218916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:51.218916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:51.218916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:51.218916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:51.218916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:51.218916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:51.218916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:51.218916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:51.218916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:51.218916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:51.220719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:51.220719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:51.220719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:51.220719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:51.220719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:51.220719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:51.220719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:51.220719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:51.220719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:51.220719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:51.220719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:51.220719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:51.220719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:51.220719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:51.220719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:51.220719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:51.220719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:51.220719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:51.222525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:51.222525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:51.222525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:51.222525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:51.222525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:51.222525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:51.222525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:51.222525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:51.222525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:01:51.222525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:51.225977 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 461/921] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 6 16:01:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:51.226582 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccd8b5-9c50-4c0b-888d-08640380f536 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:51.241595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-0fe76d56-4ff2-4cc6-b5aa-a590f78d6889 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 16:01:51.241595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:51.241595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:01:51.241595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:01:51.241595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:01:51.241595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:01:51.241595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:01:51.241595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:51.241595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 16:01:51.241595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:51.241595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:01:51.241595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:51.241595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:51.241595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:51.241595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:51.241595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:51.241595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:51.243520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:51.243520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:51.243520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:51.243520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:51.243520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:51.243520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:51.243520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:01:51.243520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:01:51.243520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:01:51.243520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:01:51.243520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:51.243520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:51.243520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:51.243520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:51.243520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:51.243520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:51.245122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:51.245122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:51.245122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:51.245122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:51.245122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:51.245122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:51.245122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:51.245122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:51.245122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:51.245122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:51.245122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:01:51.245122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:51.245122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 16:01:51.245122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:51.255536 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 461/922] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 16:01:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:51.260935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe76d56-4ff2-4cc6-b5aa-a590f78d6889 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:51.347540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:51.357376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccd8b5-9c50-4c0b-888d-08640380f536 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:51.400144 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791339b8-320d-4b48-8cfe-13496823a6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:51.400777 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791339b8-320d-4b48-8cfe-13496823a6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:51.401052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791339b8-320d-4b48-8cfe-13496823a6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:51.401334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791339b8-320d-4b48-8cfe-13496823a6c3 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:51.402077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791339b8-320d-4b48-8cfe-13496823a6c3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:51.402389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791339b8-320d-4b48-8cfe-13496823a6c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:51.403866 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791339b8-320d-4b48-8cfe-13496823a6c3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:51.430764 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 462/923] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Sep 6 16:01:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:51.431861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791339b8-320d-4b48-8cfe-13496823a6c3 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:51.462693 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe76d56-4ff2-4cc6-b5aa-a590f78d6889 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:51.529824 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec889b-c15d-4c68-b6b3-73c9ba62d4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:51.531031 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec889b-c15d-4c68-b6b3-73c9ba62d4ce None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:51.531511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec889b-c15d-4c68-b6b3-73c9ba62d4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:51.531976 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec889b-c15d-4c68-b6b3-73c9ba62d4ce None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:51.533280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec889b-c15d-4c68-b6b3-73c9ba62d4ce None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:51.533894 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec889b-c15d-4c68-b6b3-73c9ba62d4ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:51.537304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec889b-c15d-4c68-b6b3-73c9ba62d4ce None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:51.551367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 462/924] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 16:01:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:51.557938 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec889b-c15d-4c68-b6b3-73c9ba62d4ce None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:51.560211 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791339b8-320d-4b48-8cfe-13496823a6c3 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:51.603725 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7add3c-8c4d-420e-a831-d91ffef2d547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:51.604722 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7add3c-8c4d-420e-a831-d91ffef2d547 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:51.605442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7add3c-8c4d-420e-a831-d91ffef2d547 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:51.605934 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7add3c-8c4d-420e-a831-d91ffef2d547 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:51.619622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7add3c-8c4d-420e-a831-d91ffef2d547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:51.621782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7add3c-8c4d-420e-a831-d91ffef2d547 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:51.648411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-9f7add3c-8c4d-420e-a831-d91ffef2d547 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:51.648411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:51.648411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:51.648411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:51.648411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:51.648411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:51.648411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:51.648411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:51.648411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:51.648411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:51.648411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:51.648411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:51.648411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:51.648411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:51.648411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:51.648411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:51.649443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:51.649443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:51.649443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:51.649443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:51.649443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:51.649443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:51.649443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:51.649443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:51.649443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:51.649443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:51.649443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:51.649443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:51.649443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:51.649443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:51.649443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:51.649443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:51.649443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:51.650332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:51.650332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:51.650332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:51.658271 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 463/925] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:51.659883 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7add3c-8c4d-420e-a831-d91ffef2d547 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:51.664727 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:51.666905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:51.735318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec889b-c15d-4c68-b6b3-73c9ba62d4ce None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:51.779846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab935863-4d36-4111-b511-acc66a98f0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:51.780774 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab935863-4d36-4111-b511-acc66a98f0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:51.781526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab935863-4d36-4111-b511-acc66a98f0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:51.782009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab935863-4d36-4111-b511-acc66a98f0f4 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:51.782942 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab935863-4d36-4111-b511-acc66a98f0f4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:51.783464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab935863-4d36-4111-b511-acc66a98f0f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:51.785631 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab935863-4d36-4111-b511-acc66a98f0f4 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:51.802395 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 463/926] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 6 16:01:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:51.807251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab935863-4d36-4111-b511-acc66a98f0f4 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:51.854229 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7add3c-8c4d-420e-a831-d91ffef2d547 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:51.917910 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac89cd-7d76-4bb2-9cec-0ec4de1ac027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:51.919088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac89cd-7d76-4bb2-9cec-0ec4de1ac027 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:51.919635 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac89cd-7d76-4bb2-9cec-0ec4de1ac027 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:51.920179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac89cd-7d76-4bb2-9cec-0ec4de1ac027 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:51.921373 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac89cd-7d76-4bb2-9cec-0ec4de1ac027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:51.921962 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac89cd-7d76-4bb2-9cec-0ec4de1ac027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:51.924638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac89cd-7d76-4bb2-9cec-0ec4de1ac027 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:51.948514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 464/927] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:51.959479 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac89cd-7d76-4bb2-9cec-0ec4de1ac027 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:52.001358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab935863-4d36-4111-b511-acc66a98f0f4 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:52.024264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:52.060512 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18aa3f-8891-4885-9868-ee00c6647b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:52.061412 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18aa3f-8891-4885-9868-ee00c6647b53 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:52.061870 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18aa3f-8891-4885-9868-ee00c6647b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:52.062325 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18aa3f-8891-4885-9868-ee00c6647b53 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:52.063499 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18aa3f-8891-4885-9868-ee00c6647b53 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:52.065886 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18aa3f-8891-4885-9868-ee00c6647b53 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:52.067901 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:52.102145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 464/928] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:01:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:01:52.104565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e613cf1-fb41-44aa-bd91-850f879c7551 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:52.105003 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e613cf1-fb41-44aa-bd91-850f879c7551 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:52.105498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e613cf1-fb41-44aa-bd91-850f879c7551 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:52.108861 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 465/929] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:52.118554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac89cd-7d76-4bb2-9cec-0ec4de1ac027 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:52.119583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc4aab-7594-483d-9c5a-6703d81d8f18 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:52.120171 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc4aab-7594-483d-9c5a-6703d81d8f18 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:52.120881 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc4aab-7594-483d-9c5a-6703d81d8f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:52.159252 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-28dc4aab-7594-483d-9c5a-6703d81d8f18 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:52.163396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-28dc4aab-7594-483d-9c5a-6703d81d8f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:52.170799 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c31d94-aed2-4d10-af9c-5df6b54e036e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:52.171808 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c31d94-aed2-4d10-af9c-5df6b54e036e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:52.172320 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c31d94-aed2-4d10-af9c-5df6b54e036e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:52.172779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c31d94-aed2-4d10-af9c-5df6b54e036e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:52.182163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c31d94-aed2-4d10-af9c-5df6b54e036e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:52.184278 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c31d94-aed2-4d10-af9c-5df6b54e036e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:52.199046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-57c31d94-aed2-4d10-af9c-5df6b54e036e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:52.199046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:52.199046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:52.199046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:52.199046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:52.199046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:52.199046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:52.199046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:52.199046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:52.199046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:52.199046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:52.199046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:52.199046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:52.199046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:52.199046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:52.200053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:52.200053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:52.200053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:52.200053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:52.200053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:52.200053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:52.200053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:52.200053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:52.200053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:52.200053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:52.200053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:52.200053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:52.200053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:52.200053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:52.200053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:52.200053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:52.200053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:52.200950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:52.200950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:52.200950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:52.200950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:52.200950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:52.200950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:52.200950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:52.200950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:52.208045 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 465/930] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:52.209381 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c31d94-aed2-4d10-af9c-5df6b54e036e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:52.333481 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 466/931] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:52.365846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca301b-ec8a-4107-92ed-4d63e549e187 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:52.366257 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca301b-ec8a-4107-92ed-4d63e549e187 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:52.366779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca301b-ec8a-4107-92ed-4d63e549e187 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:52.369355 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 467/932] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:52.377438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4879091-1a29-4f73-bf6a-58f28f0e97e0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:52.377693 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4879091-1a29-4f73-bf6a-58f28f0e97e0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:52.377975 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4879091-1a29-4f73-bf6a-58f28f0e97e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:52.386266 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c31d94-aed2-4d10-af9c-5df6b54e036e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:52.399891 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b4879091-1a29-4f73-bf6a-58f28f0e97e0 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:52.401157 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b4879091-1a29-4f73-bf6a-58f28f0e97e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:52.427577 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:52.447803 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e5a05-4962-4e2a-94b5-60c4732b5cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:52.448463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e5a05-4962-4e2a-94b5-60c4732b5cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:52.448746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e5a05-4962-4e2a-94b5-60c4732b5cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:52.449040 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e5a05-4962-4e2a-94b5-60c4732b5cc9 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:52.449717 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e5a05-4962-4e2a-94b5-60c4732b5cc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:52.450024 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e5a05-4962-4e2a-94b5-60c4732b5cc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:52.451505 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e5a05-4962-4e2a-94b5-60c4732b5cc9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:52.467473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 466/933] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:52.478294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e5a05-4962-4e2a-94b5-60c4732b5cc9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:52.545231 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 468/934] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:52.607422 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4879091-1a29-4f73-bf6a-58f28f0e97e0 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:52.655578 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:52.676949 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e5a05-4962-4e2a-94b5-60c4732b5cc9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:52.748864 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4879091-1a29-4f73-bf6a-58f28f0e97e0 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:52.772912 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891c3e3-b94c-423a-8383-445746fca348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:52.775545 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891c3e3-b94c-423a-8383-445746fca348 None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:52.776330 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891c3e3-b94c-423a-8383-445746fca348 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:52.777306 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891c3e3-b94c-423a-8383-445746fca348 None admin] PATH_INFO: `/v3/projects/0138d62d5879495499a17e20310d4f61/users/ba58a4914ddf418db9e8b9dce5784cf8/roles/91f753ea35ce4f5a953d0a503481da1a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:52.782735 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8891c3e3-b94c-423a-8383-445746fca348 None admin] RBAC: Authorizing `identity:create_grant(project_id=0138d62d5879495499a17e20310d4f61, user_id=ba58a4914ddf418db9e8b9dce5784cf8, role_id=91f753ea35ce4f5a953d0a503481da1a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:52.784365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8891c3e3-b94c-423a-8383-445746fca348 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:52.799972 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907a7af-ba4a-49dd-a161-840615cb32a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:52.801845 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907a7af-ba4a-49dd-a161-840615cb32a1 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:52.803013 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 467/935] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:01:52 2022] PUT /identity/v3/projects/0138d62d5879495499a17e20310d4f61/users/ba58a4914ddf418db9e8b9dce5784cf8/roles/91f753ea35ce4f5a953d0a503481da1a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:01:52.803374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907a7af-ba4a-49dd-a161-840615cb32a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:52.804702 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907a7af-ba4a-49dd-a161-840615cb32a1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:52.806259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907a7af-ba4a-49dd-a161-840615cb32a1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:52.809496 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907a7af-ba4a-49dd-a161-840615cb32a1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:52.830195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-7907a7af-ba4a-49dd-a161-840615cb32a1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 16:01:52.830195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:52.830195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:01:52.830195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:01:52.830195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:01:52.830195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:01:52.830195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:01:52.830195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:52.830195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 16:01:52.830195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:52.830195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:01:52.830195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:52.830195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:52.830195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:52.830195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:52.830195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:52.832121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:52.832121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:52.832121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:52.832121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:52.832121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:52.832121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:52.832121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:52.832121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:01:52.832121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:01:52.832121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:01:52.832121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:01:52.832121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:52.832121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:52.832121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:52.832121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:52.832121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:52.832121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:52.834914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:52.834914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:52.834914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:52.834914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:52.834914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:52.834914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:52.834914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:52.834914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:52.834914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:52.834914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:52.834914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:01:52.834914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:52.834914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 16:01:52.834914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:52.839643 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 469/936] 10.222.0.26 () {56 vars in 1132 bytes} [Tue Sep 6 16:01:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:52.848034 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907a7af-ba4a-49dd-a161-840615cb32a1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:52.848330 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891c3e3-b94c-423a-8383-445746fca348 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:52.902335 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:52.980227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891c3e3-b94c-423a-8383-445746fca348 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:52.994547 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:53.007123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907a7af-ba4a-49dd-a161-840615cb32a1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:53.052706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d88d4-060a-4334-83a2-c4833dfb9696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:53.053827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d88d4-060a-4334-83a2-c4833dfb9696 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:53.054336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d88d4-060a-4334-83a2-c4833dfb9696 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:53.054797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d88d4-060a-4334-83a2-c4833dfb9696 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:53.055897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d88d4-060a-4334-83a2-c4833dfb9696 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:53.057090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff2d91-8e4c-4614-9b0f-bfed8ab34a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:53.057789 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff2d91-8e4c-4614-9b0f-bfed8ab34a7e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:53.058140 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff2d91-8e4c-4614-9b0f-bfed8ab34a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:53.058482 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff2d91-8e4c-4614-9b0f-bfed8ab34a7e None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:53.058590 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d88d4-060a-4334-83a2-c4833dfb9696 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:53.059304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff2d91-8e4c-4614-9b0f-bfed8ab34a7e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:53.059681 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff2d91-8e4c-4614-9b0f-bfed8ab34a7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:53.061136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff2d91-8e4c-4614-9b0f-bfed8ab34a7e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:53.069094 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 470/937] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 6 16:01:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:53.075293 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff2d91-8e4c-4614-9b0f-bfed8ab34a7e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:53.078510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-e51d88d4-060a-4334-83a2-c4833dfb9696 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:01:53.078510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:53.078510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:01:53.078510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:01:53.078510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:01:53.078510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:01:53.078510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:01:53.078510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:53.078510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:01:53.078510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:53.078510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:01:53.078510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:53.078510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:53.078510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:53.078510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:53.078510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:53.078510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:53.079782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:53.079782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:53.079782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:53.079782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:53.079782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:53.079782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:53.079782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:01:53.079782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:01:53.079782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:01:53.079782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:01:53.079782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:53.079782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:53.079782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:53.079782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:53.079782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:53.079782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:53.080784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:53.080784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:53.080784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:53.080784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:53.080784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:53.080784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:53.080784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:53.080784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:53.080784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:53.080784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:53.080784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:01:53.080784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:53.080784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:01:53.080784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:53.081561 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 468/938] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 6 16:01:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:53.090076 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d88d4-060a-4334-83a2-c4833dfb9696 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:53.222926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff2d91-8e4c-4614-9b0f-bfed8ab34a7e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:53.267906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d88d4-060a-4334-83a2-c4833dfb9696 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:53.272365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:53.305227 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68a76d-e0d8-453d-a6b4-938c7c6083fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:53.306235 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68a76d-e0d8-453d-a6b4-938c7c6083fc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:53.306856 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68a76d-e0d8-453d-a6b4-938c7c6083fc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:53.307350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68a76d-e0d8-453d-a6b4-938c7c6083fc None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:53.308746 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68a76d-e0d8-453d-a6b4-938c7c6083fc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:53.309425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68a76d-e0d8-453d-a6b4-938c7c6083fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:53.312234 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68a76d-e0d8-453d-a6b4-938c7c6083fc None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:53.314347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f86e5f-2ffa-4423-a8f5-697121f13a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:53.315077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f86e5f-2ffa-4423-a8f5-697121f13a55 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:53.315358 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f86e5f-2ffa-4423-a8f5-697121f13a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:53.315627 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f86e5f-2ffa-4423-a8f5-697121f13a55 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:53.316289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f86e5f-2ffa-4423-a8f5-697121f13a55 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:53.316592 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f86e5f-2ffa-4423-a8f5-697121f13a55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:53.318261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f86e5f-2ffa-4423-a8f5-697121f13a55 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:53.323259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 471/939] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 6 16:01:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:53.324412 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 469/940] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 6 16:01:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:53.330429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68a76d-e0d8-453d-a6b4-938c7c6083fc None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:53.330869 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f86e5f-2ffa-4423-a8f5-697121f13a55 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:53.380814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:53.447855 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f86e5f-2ffa-4423-a8f5-697121f13a55 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:53.485911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b654f8-7a97-4840-a9e6-dcd8407fd01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:53.486474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b654f8-7a97-4840-a9e6-dcd8407fd01b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:53.486789 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b654f8-7a97-4840-a9e6-dcd8407fd01b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:53.487068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b654f8-7a97-4840-a9e6-dcd8407fd01b None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:53.487714 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b654f8-7a97-4840-a9e6-dcd8407fd01b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:53.488018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b654f8-7a97-4840-a9e6-dcd8407fd01b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:53.489992 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b654f8-7a97-4840-a9e6-dcd8407fd01b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:53.494855 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68a76d-e0d8-453d-a6b4-938c7c6083fc None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:53.500471 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 470/941] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 6 16:01:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:53.501780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b654f8-7a97-4840-a9e6-dcd8407fd01b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:53.536581 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:53.540171 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9328e-a9d2-42df-9674-3acb196e2634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:53.541182 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9328e-a9d2-42df-9674-3acb196e2634 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:53.541654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9328e-a9d2-42df-9674-3acb196e2634 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:53.542793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9328e-a9d2-42df-9674-3acb196e2634 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:53.544304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9328e-a9d2-42df-9674-3acb196e2634 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:53.545889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9328e-a9d2-42df-9674-3acb196e2634 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:53.581875 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 472/942] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:01:53 2022] POST /identity/v3/endpoints => generated 347 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:01:53.627427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b654f8-7a97-4840-a9e6-dcd8407fd01b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:53.680383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c5ace-0a81-4dac-9ca7-41200e619cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:53.681018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c5ace-0a81-4dac-9ca7-41200e619cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:53.681300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c5ace-0a81-4dac-9ca7-41200e619cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:53.681571 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c5ace-0a81-4dac-9ca7-41200e619cb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:53.682304 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c5ace-0a81-4dac-9ca7-41200e619cb7 None admin] RBAC: Adding query filter params (interface=public, service_id=5f464edf684d4a64b687b487fca050fd, region_id=RegionOne) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:53.682612 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c5ace-0a81-4dac-9ca7-41200e619cb7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:53.684135 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c5ace-0a81-4dac-9ca7-41200e619cb7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:53.694800 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 471/943] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Sep 6 16:01:53 2022] GET /identity/v3/endpoints?service_id=5f464edf684d4a64b687b487fca050fd&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:53.697556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9328e-a9d2-42df-9674-3acb196e2634 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:53.793731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:53.825176 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9328e-a9d2-42df-9674-3acb196e2634 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:53.867360 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0deb37c-b5b5-4438-8b03-0c472481af65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:53.868781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0deb37c-b5b5-4438-8b03-0c472481af65 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:53.869516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0deb37c-b5b5-4438-8b03-0c472481af65 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:53.870209 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0deb37c-b5b5-4438-8b03-0c472481af65 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:53.871653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0deb37c-b5b5-4438-8b03-0c472481af65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:53.874687 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0deb37c-b5b5-4438-8b03-0c472481af65 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:53.891386 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 473/944] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Sep 6 16:01:53 2022] GET /identity/v3/services => generated 1497 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:01:55.593111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696c708-4c66-4f73-9ddc-ea0d19c893d8 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:55.593947 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696c708-4c66-4f73-9ddc-ea0d19c893d8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:55.594106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696c708-4c66-4f73-9ddc-ea0d19c893d8 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:55.596933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 472/945] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:55.609774 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bc18b-de32-466f-b89f-1055804fdc77 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:55.610564 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bc18b-de32-466f-b89f-1055804fdc77 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:55.610760 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bc18b-de32-466f-b89f-1055804fdc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:55.642458 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ad3bc18b-de32-466f-b89f-1055804fdc77 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:55.644429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ad3bc18b-de32-466f-b89f-1055804fdc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:55.670749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab70a3-7263-44d2-8e03-a85b52539a10 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:55.671200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab70a3-7263-44d2-8e03-a85b52539a10 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:55.671380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab70a3-7263-44d2-8e03-a85b52539a10 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:55.672945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 473/946] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:01:55.679296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59a922-6d27-423a-81f4-e44caa02d2cf None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:55.679701 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59a922-6d27-423a-81f4-e44caa02d2cf None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:55.679943 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59a922-6d27-423a-81f4-e44caa02d2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:55.699815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-bf59a922-6d27-423a-81f4-e44caa02d2cf None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:55.701159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bf59a922-6d27-423a-81f4-e44caa02d2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:55.786719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 474/947] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:01:55.788274 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 474/948] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:55.793125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d918c09-c579-4194-bdc7-00b151b37915 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:55.793978 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d918c09-c579-4194-bdc7-00b151b37915 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:55.794752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d918c09-c579-4194-bdc7-00b151b37915 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:55.797955 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 475/949] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:55.804442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e955f6a-e5cd-4f83-8dcb-d8050e903cbe None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:55.804721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e955f6a-e5cd-4f83-8dcb-d8050e903cbe None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:55.804998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e955f6a-e5cd-4f83-8dcb-d8050e903cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:55.825385 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-0e955f6a-e5cd-4f83-8dcb-d8050e903cbe None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:55.826601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0e955f6a-e5cd-4f83-8dcb-d8050e903cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:55.900431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:55.905581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 475/950] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:56.079813 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d918c09-c579-4194-bdc7-00b151b37915 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:56.107850 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:56.216222 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e955f6a-e5cd-4f83-8dcb-d8050e903cbe None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:56.219106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d918c09-c579-4194-bdc7-00b151b37915 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:56.262696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c8206-0de0-42e5-804d-54d62b9e38cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:56.263359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c8206-0de0-42e5-804d-54d62b9e38cd None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:56.263640 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c8206-0de0-42e5-804d-54d62b9e38cd None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:56.264342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c8206-0de0-42e5-804d-54d62b9e38cd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:56.278732 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c8206-0de0-42e5-804d-54d62b9e38cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:01:56.284414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-ce2c8206-0de0-42e5-804d-54d62b9e38cd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:01:56.284414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:56.284414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:56.284414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:56.284414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:56.284414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:56.284414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:56.284414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:56.284414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:56.284414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:56.284414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:56.284414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:56.284414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:01:56.284414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:01:56.284414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:01:56.284414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:01:56.287256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:01:56.287256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:01:56.287256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:01:56.287256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:01:56.287256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:56.287256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:56.287256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:56.287256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:56.287256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:56.287256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:56.287256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:56.287256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:56.287256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:56.287256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:56.287256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:56.287256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:56.288990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:56.288990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:56.288990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:56.288990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:56.288990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:01:56.288990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:01:56.288990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:01:56.288990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:01:56.288990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:01:56.288990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:01:56.288990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:01:56.288990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:56.288990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 476/951] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Sep 6 16:01:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:56.294009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c8206-0de0-42e5-804d-54d62b9e38cd None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:56.362013 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e955f6a-e5cd-4f83-8dcb-d8050e903cbe None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:56.402124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866070d7-3914-4989-bf70-2a7131e84beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:56.402809 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866070d7-3914-4989-bf70-2a7131e84beb None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:56.403092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866070d7-3914-4989-bf70-2a7131e84beb None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:56.403366 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866070d7-3914-4989-bf70-2a7131e84beb None admin] PATH_INFO: `/v3/services/compute` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:56.403991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866070d7-3914-4989-bf70-2a7131e84beb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:56.405381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866070d7-3914-4989-bf70-2a7131e84beb None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:56.415398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-866070d7-3914-4989-bf70-2a7131e84beb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 16:01:56.415398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:56.415398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:01:56.415398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:01:56.415398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:01:56.415398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:01:56.415398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:01:56.415398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:56.415398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 16:01:56.415398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:56.415398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:01:56.415398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:56.415398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:56.415398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:56.415398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:56.415398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:56.415398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:56.417624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:56.417624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:56.417624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:56.417624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:56.417624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:56.417624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:56.417624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:01:56.417624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:01:56.417624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:01:56.417624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:01:56.417624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:56.417624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:56.417624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:56.417624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:56.417624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:56.417624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:56.419508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:56.419508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:56.419508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:56.419508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:56.419508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:56.419508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:56.419508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:56.419508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:56.419508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:56.419508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:56.419508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:01:56.419508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:56.419508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 16:01:56.419508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:56.419508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 476/952] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:56.421107 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866070d7-3914-4989-bf70-2a7131e84beb None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:56.456069 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:56.497586 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c8206-0de0-42e5-804d-54d62b9e38cd None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:56.522378 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:56.549911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5514ff-5861-4d46-a5db-55d75e0db1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:56.550880 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5514ff-5861-4d46-a5db-55d75e0db1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:56.551946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5514ff-5861-4d46-a5db-55d75e0db1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:56.552221 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5514ff-5861-4d46-a5db-55d75e0db1d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:56.552903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5514ff-5861-4d46-a5db-55d75e0db1d4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:56.553216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5514ff-5861-4d46-a5db-55d75e0db1d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:56.554607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5514ff-5861-4d46-a5db-55d75e0db1d4 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:56.572616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 477/953] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Sep 6 16:01:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:56.573508 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5514ff-5861-4d46-a5db-55d75e0db1d4 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:56.580551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866070d7-3914-4989-bf70-2a7131e84beb None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:56.631452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c860e6-c833-4720-916f-6293ff1087e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:56.632576 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c860e6-c833-4720-916f-6293ff1087e4 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:56.633039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c860e6-c833-4720-916f-6293ff1087e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:56.633490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c860e6-c833-4720-916f-6293ff1087e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:56.634577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c860e6-c833-4720-916f-6293ff1087e4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:56.637099 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c860e6-c833-4720-916f-6293ff1087e4 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:56.641201 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: WARNING keystone.server.flask.common [None req-f7c860e6-c833-4720-916f-6293ff1087e4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 06 16:01:56.671874 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.sql.core [None req-f7c860e6-c833-4720-916f-6293ff1087e4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 16:01:56.671874 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 06 16:01:56.671874 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [parameters: {'id': 'a584a3eeae2749c38d3c6805d610e9e0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 06 16:01:56.671874 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81685) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 16:01:56.703036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-f7c860e6-c833-4720-916f-6293ff1087e4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a584a3eeae2749c38d3c6805d610e9e0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a584a3eeae2749c38d3c6805d610e9e0. Sep 06 16:01:56.703036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:56.703036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 16:01:56.703036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 16:01:56.703036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 16:01:56.703036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 16:01:56.703036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 16:01:56.703036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 16:01:56.703036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 16:01:56.703036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application conn.query(q) Sep 06 16:01:56.703036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 16:01:56.703036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 16:01:56.703036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 16:01:56.704109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application result.read() Sep 06 16:01:56.704109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 16:01:56.704109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 16:01:56.704109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 16:01:56.704109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 16:01:56.704109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 16:01:56.704109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 16:01:56.704109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 16:01:56.704109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 16:01:56.704109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 16:01:56.704109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:56.704109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 16:01:56.704109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:56.704109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:56.704109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 16:01:56.704109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 16:01:56.704109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 06 16:01:56.704109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 06 16:01:56.705053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 16:01:56.705053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application next(self.gen) Sep 06 16:01:56.705053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 16:01:56.705053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application yield resource Sep 06 16:01:56.705053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 16:01:56.705053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application next(self.gen) Sep 06 16:01:56.705053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 16:01:56.705053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 16:01:56.705053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 16:01:56.705053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 16:01:56.705053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 16:01:56.705053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise self.value Sep 06 16:01:56.705053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 16:01:56.705053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 16:01:56.705053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 16:01:56.705053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application session.commit() Sep 06 16:01:56.705053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 16:01:56.705966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 16:01:56.705966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 16:01:56.705966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 16:01:56.705966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 16:01:56.705966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self.session.flush() Sep 06 16:01:56.705966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 16:01:56.705966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 16:01:56.705966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 16:01:56.705966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 16:01:56.705966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 16:01:56.705966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application compat.raise_( Sep 06 16:01:56.705966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 16:01:56.705966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception Sep 06 16:01:56.705966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 16:01:56.705966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 16:01:56.705966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 16:01:56.705966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 16:01:56.706943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 16:01:56.706943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 16:01:56.706943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 16:01:56.706943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 16:01:56.706943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 06 16:01:56.706943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 06 16:01:56.706943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 16:01:56.706943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 16:01:56.706943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 16:01:56.706943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 16:01:56.706943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 16:01:56.706943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 16:01:56.706943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 16:01:56.706943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 16:01:56.706943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 16:01:56.707877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 16:01:56.707877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 16:01:56.707877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception Sep 06 16:01:56.707877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 16:01:56.707877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 16:01:56.707877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 16:01:56.707877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 16:01:56.707877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 16:01:56.707877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 16:01:56.707877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 16:01:56.707877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application conn.query(q) Sep 06 16:01:56.707877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 16:01:56.707877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 16:01:56.707877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 16:01:56.707877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application result.read() Sep 06 16:01:56.707877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 16:01:56.707877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 16:01:56.708767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 16:01:56.708767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 16:01:56.708767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 16:01:56.708767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 16:01:56.708767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 16:01:56.708767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 16:01:56.708767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 16:01:56.708767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 06 16:01:56.708767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [parameters: {'id': 'a584a3eeae2749c38d3c6805d610e9e0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 06 16:01:56.708767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 16:01:56.708767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:56.708767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:01:56.708767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:56.708767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:56.708767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 06 16:01:56.709551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 06 16:01:56.709551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 16:01:56.709551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 16:01:56.709551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 06 16:01:56.709551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:56.709551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:01:56.709551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:56.709551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:56.709551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:56.709551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:56.709551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:56.709551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:56.709551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:56.709551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:56.709551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:56.709551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:56.709551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:56.710461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:56.710461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 06 16:01:56.710461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 06 16:01:56.710461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:56.710461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:56.710461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 06 16:01:56.710461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 06 16:01:56.710461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a584a3eeae2749c38d3c6805d610e9e0. Sep 06 16:01:56.710461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:56.710461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 477/954] 10.222.0.26 () {60 vars in 1140 bytes} [Tue Sep 6 16:01:56 2022] POST /identity/v3/projects => generated 262 bytes in 287 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:56.710461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c860e6-c833-4720-916f-6293ff1087e4 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:56.736040 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5514ff-5861-4d46-a5db-55d75e0db1d4 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:56.792558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154978e-952f-4943-a015-88d105a4d3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:56.793434 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154978e-952f-4943-a015-88d105a4d3e1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:56.793891 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154978e-952f-4943-a015-88d105a4d3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:56.794337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154978e-952f-4943-a015-88d105a4d3e1 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:56.795430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154978e-952f-4943-a015-88d105a4d3e1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:56.795885 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154978e-952f-4943-a015-88d105a4d3e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:56.798280 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154978e-952f-4943-a015-88d105a4d3e1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:56.811504 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 478/955] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 6 16:01:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:56.811504 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154978e-952f-4943-a015-88d105a4d3e1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:56.826643 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:56.881186 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c860e6-c833-4720-916f-6293ff1087e4 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:56.908955 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:56.931041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05daa0-1c35-4a1c-8ce4-1bf4f88e2e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:56.931837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05daa0-1c35-4a1c-8ce4-1bf4f88e2e7f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:56.932303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05daa0-1c35-4a1c-8ce4-1bf4f88e2e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:56.932751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05daa0-1c35-4a1c-8ce4-1bf4f88e2e7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:56.943233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154978e-952f-4943-a015-88d105a4d3e1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:56.946049 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05daa0-1c35-4a1c-8ce4-1bf4f88e2e7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:56.947754 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05daa0-1c35-4a1c-8ce4-1bf4f88e2e7f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:56.963937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-3e05daa0-1c35-4a1c-8ce4-1bf4f88e2e7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:56.963937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:56.963937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:56.963937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:56.963937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:56.963937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:56.963937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:56.963937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:56.963937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:56.963937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:56.963937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:56.963937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:56.963937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:56.963937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:56.963937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:56.963937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:56.964973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:56.964973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:56.964973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:56.964973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:56.964973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:56.964973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:56.964973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:56.964973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:56.964973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:56.964973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:56.964973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:56.964973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:56.964973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:56.964973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:56.964973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:56.964973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:56.964973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:56.965863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:56.965863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:56.965863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:56.965863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 478/956] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:56.965863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05daa0-1c35-4a1c-8ce4-1bf4f88e2e7f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:56.988327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a6325-6558-4bb9-be8f-c0f060c3cb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:56.989470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a6325-6558-4bb9-be8f-c0f060c3cb5a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:56.989950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a6325-6558-4bb9-be8f-c0f060c3cb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:56.990415 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a6325-6558-4bb9-be8f-c0f060c3cb5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:57.002635 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a6325-6558-4bb9-be8f-c0f060c3cb5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:57.005450 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a6325-6558-4bb9-be8f-c0f060c3cb5a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:57.027710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-421a6325-6558-4bb9-be8f-c0f060c3cb5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:57.027710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:57.027710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:57.027710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:57.027710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:57.027710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:57.027710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:57.027710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:57.027710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:57.027710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:57.027710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:57.027710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:57.027710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:57.027710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:57.027710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:57.029602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:57.029602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:57.029602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:57.029602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:57.029602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:57.029602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:57.029602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:57.029602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:57.029602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:57.029602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:57.029602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:57.029602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:57.029602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:57.029602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:57.029602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:57.029602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:57.029602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:57.037173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:57.037173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:57.037173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:57.037173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:57.037173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:57.037173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:57.037173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:57.037173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:57.037173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 479/957] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:57.037173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54067e32-76c9-444b-8ee1-7e478b324111 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:57.037173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54067e32-76c9-444b-8ee1-7e478b324111 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:57.038361 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54067e32-76c9-444b-8ee1-7e478b324111 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:57.038361 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 480/958] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:57.038361 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54067e32-76c9-444b-8ee1-7e478b324111 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:57.075442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05daa0-1c35-4a1c-8ce4-1bf4f88e2e7f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:57.113539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573f7e7-ca21-4d62-923d-f67e8ed18d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:57.114976 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573f7e7-ca21-4d62-923d-f67e8ed18d0a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:57.115350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573f7e7-ca21-4d62-923d-f67e8ed18d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:57.115784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573f7e7-ca21-4d62-923d-f67e8ed18d0a None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:57.116882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573f7e7-ca21-4d62-923d-f67e8ed18d0a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:57.117393 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573f7e7-ca21-4d62-923d-f67e8ed18d0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:57.119677 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573f7e7-ca21-4d62-923d-f67e8ed18d0a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:57.133261 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 479/959] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 6 16:01:56 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:57.134482 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573f7e7-ca21-4d62-923d-f67e8ed18d0a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:57.149262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:57.193134 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:57.213205 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54067e32-76c9-444b-8ee1-7e478b324111 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:57.268690 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7dd6ef-a9df-46f0-9a23-84f6aa0e19bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:57.269770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dd6ef-a9df-46f0-9a23-84f6aa0e19bf None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:57.270912 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dd6ef-a9df-46f0-9a23-84f6aa0e19bf None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:57.271490 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dd6ef-a9df-46f0-9a23-84f6aa0e19bf None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:57.272563 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7dd6ef-a9df-46f0-9a23-84f6aa0e19bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:57.273068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7dd6ef-a9df-46f0-9a23-84f6aa0e19bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:57.275199 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7dd6ef-a9df-46f0-9a23-84f6aa0e19bf None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:57.292342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 481/960] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:57.299410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d58d20-d9e6-429b-abdf-bdfe96dfc2b4 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:57.299845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d58d20-d9e6-429b-abdf-bdfe96dfc2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:57.300251 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d58d20-d9e6-429b-abdf-bdfe96dfc2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:57.321541 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573f7e7-ca21-4d62-923d-f67e8ed18d0a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:57.329118 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-82d58d20-d9e6-429b-abdf-bdfe96dfc2b4 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:57.331412 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-82d58d20-d9e6-429b-abdf-bdfe96dfc2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:57.382173 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69643eb1-e650-4cf3-925e-13af995b80f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:57.383379 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69643eb1-e650-4cf3-925e-13af995b80f9 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:57.383860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69643eb1-e650-4cf3-925e-13af995b80f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:57.384344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69643eb1-e650-4cf3-925e-13af995b80f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:57.385621 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69643eb1-e650-4cf3-925e-13af995b80f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:57.386172 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69643eb1-e650-4cf3-925e-13af995b80f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:57.388967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69643eb1-e650-4cf3-925e-13af995b80f9 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:57.419179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 480/961] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 6 16:01:57 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:57.420689 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69643eb1-e650-4cf3-925e-13af995b80f9 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:57.447180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 482/962] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:57.640319 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:57.653796 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69643eb1-e650-4cf3-925e-13af995b80f9 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:57.728109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bafa4-8b8f-4a46-bfdb-119986f32c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:57.729513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bafa4-8b8f-4a46-bfdb-119986f32c20 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:57.730291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bafa4-8b8f-4a46-bfdb-119986f32c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:57.731071 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bafa4-8b8f-4a46-bfdb-119986f32c20 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:57.746336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50bafa4-8b8f-4a46-bfdb-119986f32c20 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:57.749525 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50bafa4-8b8f-4a46-bfdb-119986f32c20 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:57.776023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-b50bafa4-8b8f-4a46-bfdb-119986f32c20 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:01:57.776023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:57.776023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:57.776023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:57.776023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:57.776023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:57.776023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:57.776023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:57.776023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:57.776023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:57.776023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:57.776023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:57.776023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:01:57.776023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:01:57.776023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:01:57.776023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:01:57.778075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:57.778075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:57.778075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:01:57.778075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:57.778075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:01:57.778075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:01:57.778075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:57.778075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:57.778075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:57.778075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:57.778075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:57.778075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:57.778075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:57.778075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:57.778075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:57.778075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:57.778075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:57.778075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:57.780764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:57.780764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:57.780764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:01:57.780764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:01:57.780764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:01:57.780764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:01:57.780764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:01:57.780764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:01:57.780764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 16:01:57.780764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:57.780764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 481/963] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 6 16:01:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:57.786611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d58d20-d9e6-429b-abdf-bdfe96dfc2b4 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:57.902912 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:57.948892 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bafa4-8b8f-4a46-bfdb-119986f32c20 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:57.986785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d58d20-d9e6-429b-abdf-bdfe96dfc2b4 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:58.057587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e56c6-18c2-4285-b1de-2ea78b1f4257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:58.059039 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e56c6-18c2-4285-b1de-2ea78b1f4257 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:58.059859 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e56c6-18c2-4285-b1de-2ea78b1f4257 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:58.060504 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e56c6-18c2-4285-b1de-2ea78b1f4257 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:58.062040 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e56c6-18c2-4285-b1de-2ea78b1f4257 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:58.062832 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e56c6-18c2-4285-b1de-2ea78b1f4257 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:58.065522 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e56c6-18c2-4285-b1de-2ea78b1f4257 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:58.099886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 483/964] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Sep 6 16:01:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:58.105541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e56c6-18c2-4285-b1de-2ea78b1f4257 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:58.123193 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bafa4-8b8f-4a46-bfdb-119986f32c20 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:58.168341 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef50950-30ec-444c-b94b-3ab0c4af423e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:58.169196 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef50950-30ec-444c-b94b-3ab0c4af423e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:58.169767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef50950-30ec-444c-b94b-3ab0c4af423e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:58.170010 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef50950-30ec-444c-b94b-3ab0c4af423e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:58.170976 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef50950-30ec-444c-b94b-3ab0c4af423e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:58.173206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef50950-30ec-444c-b94b-3ab0c4af423e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:58.189472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-9ef50950-30ec-444c-b94b-3ab0c4af423e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:01:58.189472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:58.189472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:01:58.189472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:01:58.189472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:01:58.189472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:01:58.189472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:01:58.189472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:58.189472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:01:58.189472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:58.189472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:01:58.189472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:58.189472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:58.189472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:58.189472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:58.189472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:58.189472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:58.190601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:58.190601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:58.190601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:58.190601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:58.190601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:58.190601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:58.190601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:01:58.190601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:01:58.190601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:01:58.190601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:01:58.190601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:58.190601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:58.190601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:58.190601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:58.190601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:58.190601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:58.192116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:58.192116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:58.192116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:58.192116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:58.192116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:58.192116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:58.192116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:58.192116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:58.192116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:58.192116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:58.192116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:01:58.192116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:01:58.192116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:01:58.192116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:01:58.192116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 482/965] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 6 16:01:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:58.198158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef50950-30ec-444c-b94b-3ab0c4af423e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:58.242915 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:58.323130 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e56c6-18c2-4285-b1de-2ea78b1f4257 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:58.357289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:58.394189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287245e0-c5d2-4e74-83cd-c20dde3f08f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:58.395902 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287245e0-c5d2-4e74-83cd-c20dde3f08f0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:58.396667 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287245e0-c5d2-4e74-83cd-c20dde3f08f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:58.397410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287245e0-c5d2-4e74-83cd-c20dde3f08f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:58.415437 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef50950-30ec-444c-b94b-3ab0c4af423e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:58.482796 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94020b4f-bf0a-43e9-9e1a-ca0a8554deb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:58.483730 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94020b4f-bf0a-43e9-9e1a-ca0a8554deb5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:58.484270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94020b4f-bf0a-43e9-9e1a-ca0a8554deb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:58.484890 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94020b4f-bf0a-43e9-9e1a-ca0a8554deb5 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:58.486077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94020b4f-bf0a-43e9-9e1a-ca0a8554deb5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:58.486623 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94020b4f-bf0a-43e9-9e1a-ca0a8554deb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:58.489693 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94020b4f-bf0a-43e9-9e1a-ca0a8554deb5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:58.507074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 483/966] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 6 16:01:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:01:58.509758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94020b4f-bf0a-43e9-9e1a-ca0a8554deb5 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:58.542043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287245e0-c5d2-4e74-83cd-c20dde3f08f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:58.545194 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287245e0-c5d2-4e74-83cd-c20dde3f08f0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:58.668522 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 484/967] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:01:58 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 16:01:58.673992 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287245e0-c5d2-4e74-83cd-c20dde3f08f0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:58.701903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:58.702432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:58.735653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94020b4f-bf0a-43e9-9e1a-ca0a8554deb5 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:58.811164 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287245e0-c5d2-4e74-83cd-c20dde3f08f0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:58.819085 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d60b6-dc94-41a1-b5d1-fd0f366a701c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:58.820298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d60b6-dc94-41a1-b5d1-fd0f366a701c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:58.820860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d60b6-dc94-41a1-b5d1-fd0f366a701c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:58.822123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d60b6-dc94-41a1-b5d1-fd0f366a701c None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:58.823315 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d60b6-dc94-41a1-b5d1-fd0f366a701c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:58.823849 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d60b6-dc94-41a1-b5d1-fd0f366a701c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:58.827843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d60b6-dc94-41a1-b5d1-fd0f366a701c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:58.840287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 484/968] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 6 16:01:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:58.850804 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d60b6-dc94-41a1-b5d1-fd0f366a701c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:58.860212 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82599171-a3a6-42f5-8503-eef52df5765f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:58.860867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82599171-a3a6-42f5-8503-eef52df5765f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:58.861208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82599171-a3a6-42f5-8503-eef52df5765f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:58.861487 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82599171-a3a6-42f5-8503-eef52df5765f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:58.875010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82599171-a3a6-42f5-8503-eef52df5765f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:58.876642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82599171-a3a6-42f5-8503-eef52df5765f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:58.896307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-82599171-a3a6-42f5-8503-eef52df5765f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:58.896307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:58.896307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:58.896307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:58.896307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:58.896307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:58.896307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:58.896307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:58.896307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:58.896307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:58.896307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:58.896307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:58.896307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:01:58.896307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:01:58.896307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:01:58.896307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:01:58.898308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:58.898308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:58.898308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:58.898308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:58.898308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:58.898308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:58.898308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:58.898308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:58.898308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:58.898308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:58.898308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:58.898308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:58.898308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:58.898308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:58.898308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:58.898308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:58.898308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:01:58.900181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:01:58.900181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:01:58.900181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:58.900181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 485/969] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:01:58.907971 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82599171-a3a6-42f5-8503-eef52df5765f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:58.995336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:59.034459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82599171-a3a6-42f5-8503-eef52df5765f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:59.048711 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d60b6-dc94-41a1-b5d1-fd0f366a701c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:59.073344 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742854e8-e701-4189-bbc3-8c4a0b689a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:59.074402 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742854e8-e701-4189-bbc3-8c4a0b689a56 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:59.074787 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742854e8-e701-4189-bbc3-8c4a0b689a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:59.075128 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742854e8-e701-4189-bbc3-8c4a0b689a56 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:59.076238 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742854e8-e701-4189-bbc3-8c4a0b689a56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:59.076626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742854e8-e701-4189-bbc3-8c4a0b689a56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:59.078099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742854e8-e701-4189-bbc3-8c4a0b689a56 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:59.098713 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 486/970] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:59.103593 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742854e8-e701-4189-bbc3-8c4a0b689a56 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:59.115566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e89d46-9655-4bf4-9960-22820239a896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:59.116585 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e89d46-9655-4bf4-9960-22820239a896 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:59.117074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e89d46-9655-4bf4-9960-22820239a896 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:59.117580 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e89d46-9655-4bf4-9960-22820239a896 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:59.118470 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e89d46-9655-4bf4-9960-22820239a896 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:59.120122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e89d46-9655-4bf4-9960-22820239a896 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:59.155437 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 485/971] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:01:58 2022] POST /identity/v3/endpoints => generated 349 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:01:59.284190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742854e8-e701-4189-bbc3-8c4a0b689a56 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:59.320330 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:59.341930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656554c1-fc15-44cb-8283-c7ca88be2c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:59.342576 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656554c1-fc15-44cb-8283-c7ca88be2c63 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:59.342890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656554c1-fc15-44cb-8283-c7ca88be2c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:59.343163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656554c1-fc15-44cb-8283-c7ca88be2c63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:59.351576 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656554c1-fc15-44cb-8283-c7ca88be2c63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:59.353066 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656554c1-fc15-44cb-8283-c7ca88be2c63 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:59.367430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-656554c1-fc15-44cb-8283-c7ca88be2c63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:59.367430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:01:59.367430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:01:59.367430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:01:59.367430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:01:59.367430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:01:59.367430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:01:59.367430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:01:59.367430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:01:59.367430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:01:59.367430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:01:59.367430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:01:59.367430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:01:59.367430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:01:59.367430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:01:59.369332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:01:59.369332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:01:59.369332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:01:59.369332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:01:59.369332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:01:59.369332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:01:59.369332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:01:59.369332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:01:59.369332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:01:59.369332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:01:59.369332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:01:59.369332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:01:59.369332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:01:59.369332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:01:59.369332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:01:59.369332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:01:59.369332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:01:59.370494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:01:59.370494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:01:59.370494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:01:59.370494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:01:59.370494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:01:59.370494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:01:59.370494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:01:59.370494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:01:59.370494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 487/972] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:01:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:01:59.374631 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e89d46-9655-4bf4-9960-22820239a896 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:59.380948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:59.570918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e89d46-9655-4bf4-9960-22820239a896 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:59.647758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85f1c9-1e4f-4b39-b0a3-51e720e061f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:59.649337 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85f1c9-1e4f-4b39-b0a3-51e720e061f5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:59.650114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85f1c9-1e4f-4b39-b0a3-51e720e061f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:59.650860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85f1c9-1e4f-4b39-b0a3-51e720e061f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:59.652516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85f1c9-1e4f-4b39-b0a3-51e720e061f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:59.653366 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85f1c9-1e4f-4b39-b0a3-51e720e061f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:59.656493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85f1c9-1e4f-4b39-b0a3-51e720e061f5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:59.686696 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 486/973] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:01:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:01:59.693499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656554c1-fc15-44cb-8283-c7ca88be2c63 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:01:59.783758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8be58-9b4b-41d1-858f-beb1036e3e61 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:59.787462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8be58-9b4b-41d1-858f-beb1036e3e61 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:59.788245 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8be58-9b4b-41d1-858f-beb1036e3e61 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:59.792159 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 487/974] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:59 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:59.798739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3019b-2249-4efe-ae89-4b5c5c47f094 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:59.799145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3019b-2249-4efe-ae89-4b5c5c47f094 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:59.799304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3019b-2249-4efe-ae89-4b5c5c47f094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:59.824754 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-82f3019b-2249-4efe-ae89-4b5c5c47f094 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:59.826225 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-82f3019b-2249-4efe-ae89-4b5c5c47f094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:59.883981 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656554c1-fc15-44cb-8283-c7ca88be2c63 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:01:59.923818 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:01:59.937693 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 488/975] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:01:59.939427 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab51758-de07-4a57-8d3a-63a20e586a2c None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:59.939853 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab51758-de07-4a57-8d3a-63a20e586a2c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:59.940190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab51758-de07-4a57-8d3a-63a20e586a2c None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:59.947180 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 489/976] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:01:59.950139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2197e7-1666-420e-bc80-bdf388fe0256 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:59.950622 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2197e7-1666-420e-bc80-bdf388fe0256 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:59.950961 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2197e7-1666-420e-bc80-bdf388fe0256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:59.956074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2191299-b9d4-42f2-ab66-9bad14b54b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:01:59.957096 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2191299-b9d4-42f2-ab66-9bad14b54b47 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:01:59.957654 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2191299-b9d4-42f2-ab66-9bad14b54b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:01:59.958139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2191299-b9d4-42f2-ab66-9bad14b54b47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:01:59.959502 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2191299-b9d4-42f2-ab66-9bad14b54b47 None admin] RBAC: Adding query filter params (role.id=91f753ea35ce4f5a953d0a503481da1a, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=ba58a4914ddf418db9e8b9dce5784cf8) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:01:59.961087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2191299-b9d4-42f2-ab66-9bad14b54b47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:01:59.963156 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2191299-b9d4-42f2-ab66-9bad14b54b47 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:01:59.975373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6c2197e7-1666-420e-bc80-bdf388fe0256 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:01:59.976613 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6c2197e7-1666-420e-bc80-bdf388fe0256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:01:59.977897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 488/977] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:01:59 2022] GET /identity/v3/role_assignments?user.id=ba58a4914ddf418db9e8b9dce5784cf8&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=91f753ea35ce4f5a953d0a503481da1a => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:00.084874 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 490/978] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:01:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:02:00.224651 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2191299-b9d4-42f2-ab66-9bad14b54b47 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:00.304447 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:00.401585 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2191299-b9d4-42f2-ab66-9bad14b54b47 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:00.439041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2197e7-1666-420e-bc80-bdf388fe0256 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:00.471066 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc705e8-8672-4daf-bc8f-6bc8446da1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:00.472137 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc705e8-8672-4daf-bc8f-6bc8446da1f5 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:00.472505 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc705e8-8672-4daf-bc8f-6bc8446da1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:00.472835 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc705e8-8672-4daf-bc8f-6bc8446da1f5 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:00.474544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc705e8-8672-4daf-bc8f-6bc8446da1f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:00.478379 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc705e8-8672-4daf-bc8f-6bc8446da1f5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:00.516649 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 489/979] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 6 16:02:00 2022] POST /identity/v3/services => generated 242 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:00.649585 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2197e7-1666-420e-bc80-bdf388fe0256 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:00.690045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:00.712717 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc85de7-8635-41fe-ae74-a9c1c9d778e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:00.713348 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc85de7-8635-41fe-ae74-a9c1c9d778e1 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:00.713625 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc85de7-8635-41fe-ae74-a9c1c9d778e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:00.713889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc85de7-8635-41fe-ae74-a9c1c9d778e1 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:00.714554 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc85de7-8635-41fe-ae74-a9c1c9d778e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:00.716080 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc85de7-8635-41fe-ae74-a9c1c9d778e1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:00.733336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.sql.core [None req-dcc85de7-8635-41fe-ae74-a9c1c9d778e1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 16:02:00.733336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 06 16:02:00.733336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [parameters: {'user_id': '43a2a7083e9d494a9b74eed47e82609f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 06 16:02:00.733336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81685) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 16:02:00.740838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-dcc85de7-8635-41fe-ae74-a9c1c9d778e1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 06 16:02:00.740838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:00.740838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 16:02:00.740838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 16:02:00.740838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 16:02:00.740838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 16:02:00.740838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 16:02:00.740838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 16:02:00.740838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 16:02:00.740838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application conn.query(q) Sep 06 16:02:00.740838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 16:02:00.740838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 16:02:00.740838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 16:02:00.740838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application result.read() Sep 06 16:02:00.740838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 16:02:00.742367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 16:02:00.742367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 16:02:00.742367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 16:02:00.742367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 16:02:00.742367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 16:02:00.742367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 16:02:00.742367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 16:02:00.742367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 16:02:00.742367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:00.742367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 16:02:00.742367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:00.742367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:00.742367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 16:02:00.742367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 16:02:00.742367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 06 16:02:00.742367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 06 16:02:00.742367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 16:02:00.742367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application next(self.gen) Sep 06 16:02:00.744349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 16:02:00.744349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application yield resource Sep 06 16:02:00.744349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 16:02:00.744349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application next(self.gen) Sep 06 16:02:00.744349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 16:02:00.744349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 16:02:00.744349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 16:02:00.744349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 16:02:00.744349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 16:02:00.744349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise self.value Sep 06 16:02:00.744349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 16:02:00.744349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 16:02:00.744349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 16:02:00.744349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application session.commit() Sep 06 16:02:00.744349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 16:02:00.744349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 16:02:00.744349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 16:02:00.745810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 16:02:00.745810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 16:02:00.745810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self.session.flush() Sep 06 16:02:00.745810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 16:02:00.745810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 16:02:00.745810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 16:02:00.745810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 16:02:00.745810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 16:02:00.745810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application compat.raise_( Sep 06 16:02:00.745810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 16:02:00.745810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception Sep 06 16:02:00.745810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 16:02:00.745810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 16:02:00.745810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 16:02:00.745810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 16:02:00.745810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 16:02:00.745810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 16:02:00.748273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 16:02:00.748273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 16:02:00.748273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 06 16:02:00.748273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 06 16:02:00.748273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 16:02:00.748273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 16:02:00.748273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 16:02:00.748273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 16:02:00.748273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 16:02:00.748273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 16:02:00.748273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 16:02:00.748273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 16:02:00.748273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 16:02:00.748273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 16:02:00.748273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 16:02:00.748273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception Sep 06 16:02:00.749738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 16:02:00.749738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 16:02:00.749738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 16:02:00.749738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 16:02:00.749738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 16:02:00.749738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 16:02:00.749738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 16:02:00.749738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application conn.query(q) Sep 06 16:02:00.749738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 16:02:00.749738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 16:02:00.749738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 16:02:00.749738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application result.read() Sep 06 16:02:00.749738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 16:02:00.749738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 16:02:00.749738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 16:02:00.749738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 16:02:00.751338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 16:02:00.751338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 16:02:00.751338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 16:02:00.751338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 16:02:00.751338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 16:02:00.751338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 06 16:02:00.751338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [parameters: {'user_id': '43a2a7083e9d494a9b74eed47e82609f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 06 16:02:00.751338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 16:02:00.751338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:00.751338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:02:00.751338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:00.751338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:00.751338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:00.751338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:00.751338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:00.754810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:00.754810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:00.754810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:00.754810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:00.754810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:00.754810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:00.754810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:00.754810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 06 16:02:00.754810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 06 16:02:00.754810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:00.754810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:00.754810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:02:00.754810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:02:00.754810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:02:00.754810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:02:00.754810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 06 16:02:00.754810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 06 16:02:00.756634 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 06 16:02:00.756634 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 06 16:02:00.756634 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 16:02:00.756634 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 16:02:00.756634 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 06 16:02:00.756634 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:00.756634 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 491/980] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Sep 6 16:02:00 2022] POST /identity/v3/users => generated 160 bytes in 304 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:00.757852 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc705e8-8672-4daf-bc8f-6bc8446da1f5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:00.873046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc705e8-8672-4daf-bc8f-6bc8446da1f5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:00.883451 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:00.912141 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b059ee6-e9d9-4c65-9540-8d45ab79db91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:00.913143 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b059ee6-e9d9-4c65-9540-8d45ab79db91 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:00.913324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b059ee6-e9d9-4c65-9540-8d45ab79db91 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:00.913669 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b059ee6-e9d9-4c65-9540-8d45ab79db91 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:00.923542 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b059ee6-e9d9-4c65-9540-8d45ab79db91 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:00.925225 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b059ee6-e9d9-4c65-9540-8d45ab79db91 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:00.938117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-3b059ee6-e9d9-4c65-9540-8d45ab79db91 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 16:02:00.938117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:00.938117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:00.938117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:00.938117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:00.938117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:00.938117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:00.938117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:00.938117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:00.938117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:00.938117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:00.938117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:00.938117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:02:00.938117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:02:00.938117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:02:00.938117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:02:00.940252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:00.940252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:00.940252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:02:00.940252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:02:00.940252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:02:00.940252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:02:00.940252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:00.940252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:00.940252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:00.940252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:00.940252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:00.940252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:00.940252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:00.940252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:00.940252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:00.940252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:00.940252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:00.940252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:00.942868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:00.942868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:00.942868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:02:00.942868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:02:00.942868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:02:00.942868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:02:00.942868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:02:00.942868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:02:00.942868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 16:02:00.942868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:00.942868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 490/981] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:00.949061 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc85de7-8635-41fe-ae74-a9c1c9d778e1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:01.062002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc85de7-8635-41fe-ae74-a9c1c9d778e1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:01.103822 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc5127-f0c2-419b-8937-0eb5911a7bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:01.104592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc5127-f0c2-419b-8937-0eb5911a7bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:01.104962 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc5127-f0c2-419b-8937-0eb5911a7bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:01.105325 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc5127-f0c2-419b-8937-0eb5911a7bc6 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:01.106125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc5127-f0c2-419b-8937-0eb5911a7bc6 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:01.106538 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc5127-f0c2-419b-8937-0eb5911a7bc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:01.108427 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc5127-f0c2-419b-8937-0eb5911a7bc6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:01.157135 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 492/982] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 6 16:02:00 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:01.586806 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fb65b-dff7-490e-956a-7ec813127b50 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:01.587315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fb65b-dff7-490e-956a-7ec813127b50 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:01.587813 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fb65b-dff7-490e-956a-7ec813127b50 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:01.590277 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 491/983] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:01.598824 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64945bc4-5cbb-4567-91bf-833536ada84e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:01.598824 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64945bc4-5cbb-4567-91bf-833536ada84e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:01.598824 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64945bc4-5cbb-4567-91bf-833536ada84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:01.628606 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-64945bc4-5cbb-4567-91bf-833536ada84e None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:01.631611 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-64945bc4-5cbb-4567-91bf-833536ada84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:01.694972 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:01.739268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 493/984] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:02:02.018712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fb65b-dff7-490e-956a-7ec813127b50 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:02.127858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fb65b-dff7-490e-956a-7ec813127b50 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:02.189747 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2a9e9-402a-45b5-b544-f024df9ff03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:02.190864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2a9e9-402a-45b5-b544-f024df9ff03b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:02.191347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2a9e9-402a-45b5-b544-f024df9ff03b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:02.192432 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2a9e9-402a-45b5-b544-f024df9ff03b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:02.193723 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2a9e9-402a-45b5-b544-f024df9ff03b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:02.196341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2a9e9-402a-45b5-b544-f024df9ff03b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:02.206927 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4d5d0-db80-4d90-916c-786d26704a4c None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:02.207934 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4d5d0-db80-4d90-916c-786d26704a4c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:02.207934 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4d5d0-db80-4d90-916c-786d26704a4c None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:02.209053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 494/985] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:02.213989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-02e2a9e9-402a-45b5-b544-f024df9ff03b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:02:02.213989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:02.213989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:02:02.213989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:02:02.213989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:02:02.213989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:02:02.213989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:02:02.213989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:02:02.213989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:02:02.213989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:02.213989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:02:02.213989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:02.213989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:02.213989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:02.213989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:02.213989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:02.213989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:02.215014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:02.215014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:02.215014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:02.215014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:02.215014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:02.215014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:02.215014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:02:02.215014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:02:02.215014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:02:02.215014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:02:02.215014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:02.215014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:02.215014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:02.215014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:02.215014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:02.215014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:02.216290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:02.216290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:02.216290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:02.216290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:02.216290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:02.216290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:02.216290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:02.216290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:02.216290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:02.216290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:02.216290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:02:02.216290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:02:02.216290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:02:02.216290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:02.218338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f39f92-0321-46fd-9637-510a9b1a87ce None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:02.218338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f39f92-0321-46fd-9637-510a9b1a87ce None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:02.218338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f39f92-0321-46fd-9637-510a9b1a87ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:02.222201 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 492/986] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 6 16:02:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:02.226560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2a9e9-402a-45b5-b544-f024df9ff03b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:02.235373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-99f39f92-0321-46fd-9637-510a9b1a87ce None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:02.236689 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-99f39f92-0321-46fd-9637-510a9b1a87ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:02.283081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:02.343429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 495/987] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:02:02.375991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2a9e9-402a-45b5-b544-f024df9ff03b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:02.421297 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122055d-ee5b-4ab5-be0d-74b94ea96131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:02.422075 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122055d-ee5b-4ab5-be0d-74b94ea96131 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:02.422368 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122055d-ee5b-4ab5-be0d-74b94ea96131 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:02.422651 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122055d-ee5b-4ab5-be0d-74b94ea96131 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:02.423314 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122055d-ee5b-4ab5-be0d-74b94ea96131 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:02.423715 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122055d-ee5b-4ab5-be0d-74b94ea96131 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:02.425438 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122055d-ee5b-4ab5-be0d-74b94ea96131 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:02.437098 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 493/988] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 6 16:02:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:02.442813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f39f92-0321-46fd-9637-510a9b1a87ce None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:02.570025 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:02.573459 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f39f92-0321-46fd-9637-510a9b1a87ce None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:02.611256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122055d-ee5b-4ab5-be0d-74b94ea96131 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:02.618355 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174371e8-caae-411f-ad0c-1cad4ace0a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:02.618996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174371e8-caae-411f-ad0c-1cad4ace0a2c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:02.619274 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174371e8-caae-411f-ad0c-1cad4ace0a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:02.619537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174371e8-caae-411f-ad0c-1cad4ace0a2c None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:02.620174 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174371e8-caae-411f-ad0c-1cad4ace0a2c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:02.620481 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174371e8-caae-411f-ad0c-1cad4ace0a2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:02.621837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174371e8-caae-411f-ad0c-1cad4ace0a2c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:02.632651 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 496/989] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 6 16:02:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:02.634126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174371e8-caae-411f-ad0c-1cad4ace0a2c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:02.757207 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:02.766635 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122055d-ee5b-4ab5-be0d-74b94ea96131 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:02.785282 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174371e8-caae-411f-ad0c-1cad4ace0a2c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:02.818002 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ce296-6e2e-4752-8088-02e547480187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:02.818653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ce296-6e2e-4752-8088-02e547480187 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:02.818957 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ce296-6e2e-4752-8088-02e547480187 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:02.819236 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ce296-6e2e-4752-8088-02e547480187 None admin] PATH_INFO: `/v3/services/image` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:02.819820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ce296-6e2e-4752-8088-02e547480187 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:02.821303 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ce296-6e2e-4752-8088-02e547480187 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:02.831060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-666ce296-6e2e-4752-8088-02e547480187 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 16:02:02.831060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:02.831060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:02:02.831060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:02:02.831060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:02:02.831060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:02:02.831060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:02:02.831060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:02:02.831060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 16:02:02.831060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:02.831060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:02:02.831060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:02.831060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:02.831060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:02.831060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:02.831060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:02.831060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:02.832200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:02.832200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:02.832200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:02.832200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:02.832200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:02.832200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:02.832200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:02:02.832200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:02:02.832200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:02:02.832200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:02:02.832200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:02.832200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:02.832200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:02.832200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:02.832200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:02.832200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:02.836944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:02.836944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:02.836944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:02.836944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:02.836944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:02.836944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:02.836944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:02.836944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:02.836944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:02.836944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:02.836944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:02:02.836944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:02:02.836944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 16:02:02.836944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:02.836944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 494/990] 10.222.0.26 () {56 vars in 1105 bytes} [Tue Sep 6 16:02:02 2022] GET /identity/v3/services/image => generated 86 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:02.839402 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ce296-6e2e-4752-8088-02e547480187 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:02.847149 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a2f0f-c13f-4f74-ab5b-8abe85edea2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:02.847763 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a2f0f-c13f-4f74-ab5b-8abe85edea2e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:02.848057 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a2f0f-c13f-4f74-ab5b-8abe85edea2e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:02.848814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a2f0f-c13f-4f74-ab5b-8abe85edea2e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:02.849523 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a2f0f-c13f-4f74-ab5b-8abe85edea2e None admin] RBAC: Adding query filter params (interface=admin, service_id=5f464edf684d4a64b687b487fca050fd, region_id=RegionOne) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:02.849829 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a2f0f-c13f-4f74-ab5b-8abe85edea2e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:02.851279 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a2f0f-c13f-4f74-ab5b-8abe85edea2e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:02.867445 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 497/991] 10.222.0.26 () {56 vars in 1249 bytes} [Tue Sep 6 16:02:02 2022] GET /identity/v3/endpoints?service_id=5f464edf684d4a64b687b487fca050fd&interface=admin®ion_id=RegionOne => generated 193 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:02.870757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a2f0f-c13f-4f74-ab5b-8abe85edea2e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:02.910034 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:02.974027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ce296-6e2e-4752-8088-02e547480187 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:03.008997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187a8e0-0bb2-4aad-bd07-4fd37f59a88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:03.009616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187a8e0-0bb2-4aad-bd07-4fd37f59a88b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:03.009893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187a8e0-0bb2-4aad-bd07-4fd37f59a88b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:03.010165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187a8e0-0bb2-4aad-bd07-4fd37f59a88b None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:03.010919 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187a8e0-0bb2-4aad-bd07-4fd37f59a88b None admin] RBAC: Adding query filter params (name=image) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:03.011223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187a8e0-0bb2-4aad-bd07-4fd37f59a88b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:03.012647 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187a8e0-0bb2-4aad-bd07-4fd37f59a88b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:03.018165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 495/992] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:03.019465 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed5904-180a-4276-bae7-1148ec85d626 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:03.020753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed5904-180a-4276-bae7-1148ec85d626 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:03.021007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed5904-180a-4276-bae7-1148ec85d626 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:03.022418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 496/993] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:03.025018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed5904-180a-4276-bae7-1148ec85d626 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:03.036567 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a2f0f-c13f-4f74-ab5b-8abe85edea2e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:03.057626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:03.075403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fde2f-4880-4bbb-ab05-94e848dede4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:03.075934 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fde2f-4880-4bbb-ab05-94e848dede4d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:03.076216 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fde2f-4880-4bbb-ab05-94e848dede4d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:03.076482 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fde2f-4880-4bbb-ab05-94e848dede4d None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:03.077091 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8fde2f-4880-4bbb-ab05-94e848dede4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:03.078474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8fde2f-4880-4bbb-ab05-94e848dede4d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:03.095026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 498/994] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Sep 6 16:02:02 2022] GET /identity/v3/services => generated 1727 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:02:03.095026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a4a0f-6517-4ad5-a954-0fbd0053c195 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:03.095026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a4a0f-6517-4ad5-a954-0fbd0053c195 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:03.095026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a4a0f-6517-4ad5-a954-0fbd0053c195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:03.116470 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-bf9a4a0f-6517-4ad5-a954-0fbd0053c195 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:03.118288 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bf9a4a0f-6517-4ad5-a954-0fbd0053c195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:03.171726 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed5904-180a-4276-bae7-1148ec85d626 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:03.233047 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b7809-b85c-4d0b-bf18-ad1cb299afa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:03.233688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b7809-b85c-4d0b-bf18-ad1cb299afa8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:03.233969 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b7809-b85c-4d0b-bf18-ad1cb299afa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:03.234245 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b7809-b85c-4d0b-bf18-ad1cb299afa8 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:03.234926 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b7809-b85c-4d0b-bf18-ad1cb299afa8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:03.235234 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b7809-b85c-4d0b-bf18-ad1cb299afa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:03.236648 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b7809-b85c-4d0b-bf18-ad1cb299afa8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:03.239240 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 499/995] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:03.243774 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 497/996] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:03 2022] GET /identity/v3/services?type=image => generated 122 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:03.515424 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a4a0f-6517-4ad5-a954-0fbd0053c195 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:03.544886 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:03.649433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a4a0f-6517-4ad5-a954-0fbd0053c195 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:03.699683 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f034f-18a4-4736-9a38-d7590b5e1e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:03.700311 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f034f-18a4-4736-9a38-d7590b5e1e33 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:03.700582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f034f-18a4-4736-9a38-d7590b5e1e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:03.700849 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f034f-18a4-4736-9a38-d7590b5e1e33 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:03.701428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f034f-18a4-4736-9a38-d7590b5e1e33 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:03.702820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f034f-18a4-4736-9a38-d7590b5e1e33 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:03.713339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-ea8f034f-18a4-4736-9a38-d7590b5e1e33 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 16:02:03.713339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:03.713339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:02:03.713339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:02:03.713339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:02:03.713339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:02:03.713339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:02:03.713339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:02:03.713339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 16:02:03.713339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:03.713339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:02:03.713339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:03.713339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:03.713339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:03.713339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:03.713339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:03.713339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:03.719259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:03.719259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:03.719259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:03.719259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:03.719259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:03.719259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:03.719259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:02:03.719259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:02:03.719259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:02:03.719259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:02:03.719259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:03.719259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:03.719259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:03.719259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:03.719259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:03.719259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:03.720124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:03.720124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:03.720124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:03.720124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:03.720124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:03.720124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:03.720124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:03.720124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:03.720124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:03.720124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:03.720124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:02:03.720124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:02:03.720124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 16:02:03.720124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:03.720124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 500/997] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:02:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:02:03.725326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b7809-b85c-4d0b-bf18-ad1cb299afa8 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:03.847867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:03.866410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b7809-b85c-4d0b-bf18-ad1cb299afa8 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:03.928188 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e416cb-2953-402c-985f-087a9c72536e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:03.929378 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e416cb-2953-402c-985f-087a9c72536e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:03.929872 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e416cb-2953-402c-985f-087a9c72536e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:03.930358 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e416cb-2953-402c-985f-087a9c72536e None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:03.931444 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e416cb-2953-402c-985f-087a9c72536e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:03.931985 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e416cb-2953-402c-985f-087a9c72536e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:03.934503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e416cb-2953-402c-985f-087a9c72536e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:03.950502 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 498/998] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 6 16:02:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:03.952452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f034f-18a4-4736-9a38-d7590b5e1e33 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:04.092896 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:04.121402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f034f-18a4-4736-9a38-d7590b5e1e33 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:04.167413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7279d7a-9025-48d9-94c6-8d499c7061db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:04.168031 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7279d7a-9025-48d9-94c6-8d499c7061db None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:04.168310 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7279d7a-9025-48d9-94c6-8d499c7061db None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:04.168569 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7279d7a-9025-48d9-94c6-8d499c7061db None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:04.169206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7279d7a-9025-48d9-94c6-8d499c7061db None admin] RBAC: Adding query filter params (type=compute) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:04.169502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7279d7a-9025-48d9-94c6-8d499c7061db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:04.170913 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7279d7a-9025-48d9-94c6-8d499c7061db None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:04.177343 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 501/999] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 6 16:02:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:04.183758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e416cb-2953-402c-985f-087a9c72536e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:04.303809 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e416cb-2953-402c-985f-087a9c72536e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:04.347257 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0755f52-47c0-40d1-8644-d54153eded4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:04.347866 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0755f52-47c0-40d1-8644-d54153eded4c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:04.348681 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0755f52-47c0-40d1-8644-d54153eded4c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:04.349067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0755f52-47c0-40d1-8644-d54153eded4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:04.350041 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0755f52-47c0-40d1-8644-d54153eded4c None admin] RBAC: Adding query filter params (interface=public, service_id=2e4cb3e17b1d4c9cbd301fe94d4dacdf, region_id=RegionOne) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:04.350479 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0755f52-47c0-40d1-8644-d54153eded4c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:04.352590 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0755f52-47c0-40d1-8644-d54153eded4c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:04.359495 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 499/1000] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Sep 6 16:02:04 2022] GET /identity/v3/endpoints?service_id=2e4cb3e17b1d4c9cbd301fe94d4dacdf&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:04.365755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7279d7a-9025-48d9-94c6-8d499c7061db None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:04.483195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7279d7a-9025-48d9-94c6-8d499c7061db None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:04.519736 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7947c-932b-4d05-980a-9252acec98b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:04.520488 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7947c-932b-4d05-980a-9252acec98b4 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:04.520953 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7947c-932b-4d05-980a-9252acec98b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:04.521367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7947c-932b-4d05-980a-9252acec98b4 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:04.521984 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7947c-932b-4d05-980a-9252acec98b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:04.523615 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7947c-932b-4d05-980a-9252acec98b4 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:04.530750 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 502/1001] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Sep 6 16:02:04 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:02:05.226649 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de798b-19a5-4df7-a2d9-113628a8c8d2 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:05.227262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de798b-19a5-4df7-a2d9-113628a8c8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:05.227262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de798b-19a5-4df7-a2d9-113628a8c8d2 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:05.228718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 500/1002] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:05.234003 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebdc9c-da61-45e3-b7ae-061d40cf7175 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:05.234290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebdc9c-da61-45e3-b7ae-061d40cf7175 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:05.234567 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebdc9c-da61-45e3-b7ae-061d40cf7175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:05.253641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-80ebdc9c-da61-45e3-b7ae-061d40cf7175 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:05.255542 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-80ebdc9c-da61-45e3-b7ae-061d40cf7175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:05.256049 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7967e56-ec25-48e9-974c-4fc3908d3ac2 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:05.256338 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7967e56-ec25-48e9-974c-4fc3908d3ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:05.256616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7967e56-ec25-48e9-974c-4fc3908d3ac2 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:05.258277 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 501/1003] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:05.263710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3967eea-b2c0-48f8-af62-d6bcefb67400 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:05.264395 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3967eea-b2c0-48f8-af62-d6bcefb67400 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:05.264395 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3967eea-b2c0-48f8-af62-d6bcefb67400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:05.266943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:05.284816 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e3967eea-b2c0-48f8-af62-d6bcefb67400 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:05.286148 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e3967eea-b2c0-48f8-af62-d6bcefb67400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:05.298288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:05.336887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 503/1004] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:05 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:05.366698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 502/1005] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:05 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:02:05.611807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebdc9c-da61-45e3-b7ae-061d40cf7175 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:05.634188 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3967eea-b2c0-48f8-af62-d6bcefb67400 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:05.725737 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebdc9c-da61-45e3-b7ae-061d40cf7175 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:05.760829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3967eea-b2c0-48f8-af62-d6bcefb67400 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:05.767036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078169e-f6dd-44f3-8469-539a0d30d404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:05.767711 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078169e-f6dd-44f3-8469-539a0d30d404 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:05.767990 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078169e-f6dd-44f3-8469-539a0d30d404 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:05.768264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078169e-f6dd-44f3-8469-539a0d30d404 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:05.768847 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078169e-f6dd-44f3-8469-539a0d30d404 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:05.770912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078169e-f6dd-44f3-8469-539a0d30d404 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:05.780236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-4078169e-f6dd-44f3-8469-539a0d30d404 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:02:05.780236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:05.780236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:02:05.780236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:02:05.780236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:02:05.780236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:02:05.780236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:02:05.780236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:02:05.780236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:02:05.780236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:05.780236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:02:05.780236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:05.780236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:05.780236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:05.780236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:05.780236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:05.780236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:05.781283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:05.781283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:05.781283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:05.781283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:05.781283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:05.781283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:05.781283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:02:05.781283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:02:05.781283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:02:05.781283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:02:05.781283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:05.781283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:05.781283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:05.781283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:05.781283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:05.781283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:05.782115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:05.782115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:05.782115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:05.782115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:05.782115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:05.782115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:05.782115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:05.782115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:05.782115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:05.782115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:05.782115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:02:05.782115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:02:05.782115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 16:02:05.782115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:05.782115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 504/1006] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 6 16:02:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:02:05.787848 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078169e-f6dd-44f3-8469-539a0d30d404 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:05.802345 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:05.806021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d4552-d72e-417c-94a7-aa1085065830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:05.806701 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d4552-d72e-417c-94a7-aa1085065830 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:05.807031 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d4552-d72e-417c-94a7-aa1085065830 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:05.807291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d4552-d72e-417c-94a7-aa1085065830 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:05.808003 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d4552-d72e-417c-94a7-aa1085065830 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:05.809413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d4552-d72e-417c-94a7-aa1085065830 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:05.830746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 503/1007] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 6 16:02:05 2022] POST /identity/v3/services => generated 242 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:02:05.909861 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078169e-f6dd-44f3-8469-539a0d30d404 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:05.955994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4041dc8-55c4-488b-9099-8cb3d1247715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:05.956611 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4041dc8-55c4-488b-9099-8cb3d1247715 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:05.956887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4041dc8-55c4-488b-9099-8cb3d1247715 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:05.957158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4041dc8-55c4-488b-9099-8cb3d1247715 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:05.958110 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4041dc8-55c4-488b-9099-8cb3d1247715 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:05.958426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4041dc8-55c4-488b-9099-8cb3d1247715 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:05.960005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4041dc8-55c4-488b-9099-8cb3d1247715 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:05.970843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 505/1008] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 6 16:02:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:05.971761 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d4552-d72e-417c-94a7-aa1085065830 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:05.982090 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:06.091691 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d4552-d72e-417c-94a7-aa1085065830 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:06.127293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabee723-8060-461a-a233-7d5c7206519f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:06.128161 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabee723-8060-461a-a233-7d5c7206519f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:06.128618 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabee723-8060-461a-a233-7d5c7206519f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:06.129044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabee723-8060-461a-a233-7d5c7206519f None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:06.129965 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabee723-8060-461a-a233-7d5c7206519f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:06.130437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabee723-8060-461a-a233-7d5c7206519f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:06.131979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabee723-8060-461a-a233-7d5c7206519f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:06.137756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 504/1009] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 6 16:02:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:06.147081 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4041dc8-55c4-488b-9099-8cb3d1247715 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:06.220561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:06.257841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4041dc8-55c4-488b-9099-8cb3d1247715 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:06.293541 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc8a87-0a3d-4866-a539-c5dfb426b7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:06.294158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc8a87-0a3d-4866-a539-c5dfb426b7b7 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:06.294436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc8a87-0a3d-4866-a539-c5dfb426b7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:06.294745 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc8a87-0a3d-4866-a539-c5dfb426b7b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:06.295431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc8a87-0a3d-4866-a539-c5dfb426b7b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:06.296794 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc8a87-0a3d-4866-a539-c5dfb426b7b7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:06.315429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 506/1010] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:02:06 2022] POST /identity/v3/endpoints => generated 325 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:06.533371 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb31f8e-a5e9-4540-bf0b-6719f045370d None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:06.533860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb31f8e-a5e9-4540-bf0b-6719f045370d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:06.533860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb31f8e-a5e9-4540-bf0b-6719f045370d None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:06.535516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 505/1011] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:06.541560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d7693-5d99-4300-b360-6db64bc5b5ff None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:06.541837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d7693-5d99-4300-b360-6db64bc5b5ff None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:06.542111 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d7693-5d99-4300-b360-6db64bc5b5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:06.562876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-774d7693-5d99-4300-b360-6db64bc5b5ff None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:06.564098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-774d7693-5d99-4300-b360-6db64bc5b5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:06.655984 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 507/1012] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:06 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:06.950901 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb31f8e-a5e9-4540-bf0b-6719f045370d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:07.030460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:07.069706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb31f8e-a5e9-4540-bf0b-6719f045370d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:07.113148 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c968139-0c65-41e8-b987-1d8a33d2e8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:07.114952 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c968139-0c65-41e8-b987-1d8a33d2e8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:07.115288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c968139-0c65-41e8-b987-1d8a33d2e8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:07.115617 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c968139-0c65-41e8-b987-1d8a33d2e8f3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:07.116195 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c968139-0c65-41e8-b987-1d8a33d2e8f3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:07.118119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c968139-0c65-41e8-b987-1d8a33d2e8f3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:07.133059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-2c968139-0c65-41e8-b987-1d8a33d2e8f3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 16:02:07.133059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:07.133059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:02:07.133059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:02:07.133059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:02:07.133059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:02:07.133059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:02:07.133059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:02:07.133059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 16:02:07.133059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:07.133059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:02:07.133059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:07.133059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:07.133059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:07.133059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:07.133059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:07.133059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:07.135004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:07.135004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:07.135004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:07.135004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:07.135004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:07.135004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:07.135004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:02:07.135004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:02:07.135004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:02:07.135004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:02:07.135004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:07.135004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:07.135004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:07.135004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:07.135004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:07.135004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:07.136321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:07.136321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:07.136321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:07.136321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:07.136321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:07.136321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:07.136321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:07.136321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:07.136321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:07.136321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:07.136321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:02:07.136321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:02:07.136321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 16:02:07.136321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:07.139160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 506/1013] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:02:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:07.142544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d7693-5d99-4300-b360-6db64bc5b5ff None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:07.245305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:07.325162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d7693-5d99-4300-b360-6db64bc5b5ff None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:07.389432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ed09d-5091-4b24-8d74-56d966c320dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:07.390453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ed09d-5091-4b24-8d74-56d966c320dc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:07.390975 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ed09d-5091-4b24-8d74-56d966c320dc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:07.391483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ed09d-5091-4b24-8d74-56d966c320dc None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:07.392664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ed09d-5091-4b24-8d74-56d966c320dc None admin] RBAC: Adding query filter params (name=compute) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:07.393204 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ed09d-5091-4b24-8d74-56d966c320dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:07.396189 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ed09d-5091-4b24-8d74-56d966c320dc None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:07.404417 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 508/1014] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 6 16:02:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:07.414366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c968139-0c65-41e8-b987-1d8a33d2e8f3 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:07.584149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c968139-0c65-41e8-b987-1d8a33d2e8f3 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:07.644243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54955791-b6dd-4fe5-b799-52445ebd987f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:07.645632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54955791-b6dd-4fe5-b799-52445ebd987f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:07.646085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54955791-b6dd-4fe5-b799-52445ebd987f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:07.646572 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54955791-b6dd-4fe5-b799-52445ebd987f None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:07.647773 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54955791-b6dd-4fe5-b799-52445ebd987f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:07.648316 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54955791-b6dd-4fe5-b799-52445ebd987f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:07.650914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54955791-b6dd-4fe5-b799-52445ebd987f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:07.653506 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:07.666928 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 507/1015] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 6 16:02:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:07.668986 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ed09d-5091-4b24-8d74-56d966c320dc None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:07.797876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ed09d-5091-4b24-8d74-56d966c320dc None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:07.822056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:07.845870 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f16de-ae84-4f1a-a809-69abd4172eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:07.846386 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f16de-ae84-4f1a-a809-69abd4172eeb None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:07.846712 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f16de-ae84-4f1a-a809-69abd4172eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:07.846991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f16de-ae84-4f1a-a809-69abd4172eeb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:07.847611 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f16de-ae84-4f1a-a809-69abd4172eeb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:07.849244 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f16de-ae84-4f1a-a809-69abd4172eeb None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:07.881195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 509/1016] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:02:07 2022] POST /identity/v3/endpoints => generated 334 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:07.996286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cabc7-0020-4925-98fd-82e21b4c3705 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:07.996553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cabc7-0020-4925-98fd-82e21b4c3705 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:07.996988 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cabc7-0020-4925-98fd-82e21b4c3705 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:07.998491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 508/1017] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:08.004218 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cce474-eb52-4e3a-a693-ee1d1056c416 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:08.005420 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cce474-eb52-4e3a-a693-ee1d1056c416 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:08.005697 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cce474-eb52-4e3a-a693-ee1d1056c416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:08.031053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-20cce474-eb52-4e3a-a693-ee1d1056c416 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:08.033252 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-20cce474-eb52-4e3a-a693-ee1d1056c416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:08.161195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 510/1018] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:08.343627 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d6239-9e9f-49b3-ac46-6d951abd93f8 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:08.343892 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d6239-9e9f-49b3-ac46-6d951abd93f8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:08.344214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d6239-9e9f-49b3-ac46-6d951abd93f8 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:08.345796 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 509/1019] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:08.351843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6bbda-a4a3-4ec2-827a-19900e42a9ed None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:08.352236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6bbda-a4a3-4ec2-827a-19900e42a9ed None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:08.352397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6bbda-a4a3-4ec2-827a-19900e42a9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:08.372055 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-bbc6bbda-a4a3-4ec2-827a-19900e42a9ed None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:08.373265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bbc6bbda-a4a3-4ec2-827a-19900e42a9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:08.440982 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d6239-9e9f-49b3-ac46-6d951abd93f8 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:08.450841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 511/1020] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:08.570774 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d6239-9e9f-49b3-ac46-6d951abd93f8 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:08.626769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e00a7-9959-4808-b7f6-43f1a0c2ec2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:08.627777 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e00a7-9959-4808-b7f6-43f1a0c2ec2a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:08.627921 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e00a7-9959-4808-b7f6-43f1a0c2ec2a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:08.628314 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e00a7-9959-4808-b7f6-43f1a0c2ec2a None admin] PATH_INFO: `/v3/services/image` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:08.628751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e00a7-9959-4808-b7f6-43f1a0c2ec2a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:08.632192 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e00a7-9959-4808-b7f6-43f1a0c2ec2a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:08.644222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-9d6e00a7-9959-4808-b7f6-43f1a0c2ec2a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 16:02:08.644222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:08.644222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:02:08.644222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:02:08.644222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:02:08.644222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:02:08.644222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:02:08.644222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:02:08.644222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 16:02:08.644222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:08.644222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:02:08.644222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:08.644222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:08.644222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:08.644222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:08.644222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:08.644222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:08.647138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:08.647138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:08.647138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:08.647138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:08.647138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:08.647138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:08.647138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:02:08.647138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:02:08.647138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:02:08.647138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:02:08.647138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:08.647138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:08.647138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:08.647138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:08.647138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:08.647138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:08.648672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:08.648672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:08.648672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:08.648672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:08.648672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:08.648672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:08.648672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:08.648672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:08.648672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:08.648672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:08.648672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:02:08.648672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:02:08.648672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 16:02:08.648672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:08.648672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 510/1021] 10.222.0.26 () {56 vars in 1105 bytes} [Tue Sep 6 16:02:08 2022] GET /identity/v3/services/image => generated 86 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:02:08.652708 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6bbda-a4a3-4ec2-827a-19900e42a9ed None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:08.700894 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:08.724836 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e00a7-9959-4808-b7f6-43f1a0c2ec2a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:08.795889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6bbda-a4a3-4ec2-827a-19900e42a9ed None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:08.837373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d203fa-0bb3-495d-a8ab-27145c893c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:08.837743 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:08.838715 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d203fa-0bb3-495d-a8ab-27145c893c92 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:08.840495 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d203fa-0bb3-495d-a8ab-27145c893c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:08.840495 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d203fa-0bb3-495d-a8ab-27145c893c92 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:08.840495 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d203fa-0bb3-495d-a8ab-27145c893c92 None admin] RBAC: Adding query filter params (name=image) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:08.840495 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d203fa-0bb3-495d-a8ab-27145c893c92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:08.842139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d203fa-0bb3-495d-a8ab-27145c893c92 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:08.854950 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 512/1022] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:08.856710 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d203fa-0bb3-495d-a8ab-27145c893c92 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:08.894577 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e00a7-9959-4808-b7f6-43f1a0c2ec2a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:08.956556 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66758af-268f-4f25-a1d3-63d4c35663ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:08.957422 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66758af-268f-4f25-a1d3-63d4c35663ea None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:08.957804 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66758af-268f-4f25-a1d3-63d4c35663ea None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:08.958271 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66758af-268f-4f25-a1d3-63d4c35663ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:08.960204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66758af-268f-4f25-a1d3-63d4c35663ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:08.962406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66758af-268f-4f25-a1d3-63d4c35663ea None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:08.966729 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 511/1023] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:08.972571 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66758af-268f-4f25-a1d3-63d4c35663ea None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:09.026960 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:09.039754 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d203fa-0bb3-495d-a8ab-27145c893c92 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:09.098693 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b2395-acab-472e-94ec-f9e367441095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:09.099330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b2395-acab-472e-94ec-f9e367441095 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:09.099605 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b2395-acab-472e-94ec-f9e367441095 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:09.099873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b2395-acab-472e-94ec-f9e367441095 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:09.100509 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b2395-acab-472e-94ec-f9e367441095 None admin] RBAC: Adding query filter params (type=image) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:09.100808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b2395-acab-472e-94ec-f9e367441095 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:09.102173 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b2395-acab-472e-94ec-f9e367441095 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:09.108856 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 513/1024] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:09.118646 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b2395-acab-472e-94ec-f9e367441095 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:09.156987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66758af-268f-4f25-a1d3-63d4c35663ea None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:09.203202 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:09.214061 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236cf54-db5d-4025-afa9-d33358113725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:09.214970 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236cf54-db5d-4025-afa9-d33358113725 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:09.215416 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236cf54-db5d-4025-afa9-d33358113725 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:09.215861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236cf54-db5d-4025-afa9-d33358113725 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:09.216952 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236cf54-db5d-4025-afa9-d33358113725 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:09.219283 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236cf54-db5d-4025-afa9-d33358113725 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:09.261044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 512/1025] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 6 16:02:08 2022] POST /identity/v3/projects => generated 311 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:09.289865 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b2395-acab-472e-94ec-f9e367441095 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:09.351000 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5568324-08dd-4a8a-bfa7-dedfa5d7c55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:09.352019 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5568324-08dd-4a8a-bfa7-dedfa5d7c55d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:09.352529 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5568324-08dd-4a8a-bfa7-dedfa5d7c55d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:09.353028 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5568324-08dd-4a8a-bfa7-dedfa5d7c55d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:09.354379 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5568324-08dd-4a8a-bfa7-dedfa5d7c55d None admin] RBAC: Adding query filter params (interface=public, service_id=e98090fc14104b8aa24017d93d3af5af, region_id=RegionOne) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:09.354961 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5568324-08dd-4a8a-bfa7-dedfa5d7c55d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:09.357582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5568324-08dd-4a8a-bfa7-dedfa5d7c55d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:09.367803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 514/1026] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Sep 6 16:02:09 2022] GET /identity/v3/endpoints?service_id=e98090fc14104b8aa24017d93d3af5af&interface=public®ion_id=RegionOne => generated 194 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:09.375288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236cf54-db5d-4025-afa9-d33358113725 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:09.514249 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236cf54-db5d-4025-afa9-d33358113725 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:09.556098 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2360b0b-c4d6-44a8-bbcd-f0c4c25386cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:09.556721 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2360b0b-c4d6-44a8-bbcd-f0c4c25386cb None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:09.557055 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2360b0b-c4d6-44a8-bbcd-f0c4c25386cb None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:09.557375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2360b0b-c4d6-44a8-bbcd-f0c4c25386cb None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:09.558109 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2360b0b-c4d6-44a8-bbcd-f0c4c25386cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:09.559529 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2360b0b-c4d6-44a8-bbcd-f0c4c25386cb None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:09.570798 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 513/1027] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Sep 6 16:02:09 2022] GET /identity/v3/services => generated 1957 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:02:10.108552 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c963d-ab21-4d9d-900c-36d661a4bfea None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:10.109096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c963d-ab21-4d9d-900c-36d661a4bfea None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:10.109096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c963d-ab21-4d9d-900c-36d661a4bfea None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:10.111024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 515/1028] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:10.117014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaef7f4-2778-455a-b214-049b02616735 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:10.117549 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaef7f4-2778-455a-b214-049b02616735 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:10.117549 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaef7f4-2778-455a-b214-049b02616735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:10.138964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0aaef7f4-2778-455a-b214-049b02616735 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:10.140218 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0aaef7f4-2778-455a-b214-049b02616735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:10.206633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:10.255432 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 514/1029] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:10.644466 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c963d-ab21-4d9d-900c-36d661a4bfea None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:10.675657 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:10.787946 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c963d-ab21-4d9d-900c-36d661a4bfea None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:10.831554 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b6887-8efd-4860-83be-d4255978b43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:10.832446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b6887-8efd-4860-83be-d4255978b43a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:10.832865 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b6887-8efd-4860-83be-d4255978b43a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:10.833445 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b6887-8efd-4860-83be-d4255978b43a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:10.845554 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b6887-8efd-4860-83be-d4255978b43a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:10.851721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-7a5b6887-8efd-4860-83be-d4255978b43a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:02:10.851721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:10.851721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:10.851721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:10.851721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:10.851721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:10.851721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:10.851721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:10.851721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:10.851721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:10.851721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:10.851721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:10.851721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:10.851721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:10.851721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:10.851721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:10.853487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:10.853487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:10.853487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:10.853487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:10.853487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:10.853487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:10.853487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:10.853487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:10.853487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:10.853487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:10.853487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:10.853487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:10.853487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:10.853487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:10.853487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:10.853487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:10.857752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:10.857752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:10.857752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:10.857752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:10.857752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:10.857752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:10.857752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:10.857752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:10.857752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:10.857752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:10.857752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:02:10.857752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:10.859372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 516/1030] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Sep 6 16:02:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:02:10.862088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaef7f4-2778-455a-b214-049b02616735 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:10.983224 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaef7f4-2778-455a-b214-049b02616735 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:11.042600 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497ea93-2a07-4607-bb9b-e96d5a25b180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:11.043678 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497ea93-2a07-4607-bb9b-e96d5a25b180 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:11.044184 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497ea93-2a07-4607-bb9b-e96d5a25b180 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:11.044686 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497ea93-2a07-4607-bb9b-e96d5a25b180 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:11.045732 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497ea93-2a07-4607-bb9b-e96d5a25b180 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:11.046291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497ea93-2a07-4607-bb9b-e96d5a25b180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:11.048933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497ea93-2a07-4607-bb9b-e96d5a25b180 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:11.064508 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 515/1031] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Sep 6 16:02:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:11.073805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b6887-8efd-4860-83be-d4255978b43a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:11.192941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:11.216190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b6887-8efd-4860-83be-d4255978b43a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:11.257189 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a04d0d-36ce-4619-93f1-14a993387aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:11.257889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a04d0d-36ce-4619-93f1-14a993387aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:11.258169 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a04d0d-36ce-4619-93f1-14a993387aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:11.258441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a04d0d-36ce-4619-93f1-14a993387aa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:11.271968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a04d0d-36ce-4619-93f1-14a993387aa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:11.273448 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a04d0d-36ce-4619-93f1-14a993387aa7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:11.290375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-89a04d0d-36ce-4619-93f1-14a993387aa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:02:11.290375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:11.290375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:11.290375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:11.290375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:11.290375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:11.290375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:11.290375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:11.290375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:11.290375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:11.290375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:11.290375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:11.290375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:02:11.290375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:02:11.290375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:02:11.290375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:02:11.292436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:11.292436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:11.292436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:11.292436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:11.292436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:11.292436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:11.292436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:11.292436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:11.292436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:11.292436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:11.292436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:11.292436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:11.292436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:11.292436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:11.292436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:11.292436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:11.292436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:02:11.294653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:02:11.294653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:02:11.294653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:11.296773 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 517/1032] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:11.297554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497ea93-2a07-4607-bb9b-e96d5a25b180 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:11.345381 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:11.416969 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497ea93-2a07-4607-bb9b-e96d5a25b180 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:11.453664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcebcc8-894f-46d8-a637-4ba48532ae97 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:11.453842 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcebcc8-894f-46d8-a637-4ba48532ae97 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:11.454128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcebcc8-894f-46d8-a637-4ba48532ae97 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:11.455682 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 518/1033] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:11.462579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00ddc3-ec02-41ba-8498-1175d2bf27b6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:11.463022 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00ddc3-ec02-41ba-8498-1175d2bf27b6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:11.463424 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00ddc3-ec02-41ba-8498-1175d2bf27b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:11.482973 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa95e93-c442-4e27-8670-d49f39c6deb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:11.483609 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa95e93-c442-4e27-8670-d49f39c6deb9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:11.483881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa95e93-c442-4e27-8670-d49f39c6deb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:11.484153 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa95e93-c442-4e27-8670-d49f39c6deb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:11.484804 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa95e93-c442-4e27-8670-d49f39c6deb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:11.485106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa95e93-c442-4e27-8670-d49f39c6deb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:11.486546 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa95e93-c442-4e27-8670-d49f39c6deb9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:11.499486 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6b00ddc3-ec02-41ba-8498-1175d2bf27b6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:11.500666 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6b00ddc3-ec02-41ba-8498-1175d2bf27b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:11.504784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 516/1034] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:11.511699 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa95e93-c442-4e27-8670-d49f39c6deb9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:11.622998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 519/1035] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:02:11.637481 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce739fd7-eddf-419c-bf97-e9c0134e7235 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:11.638293 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce739fd7-eddf-419c-bf97-e9c0134e7235 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:11.638575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce739fd7-eddf-419c-bf97-e9c0134e7235 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:11.640498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 520/1036] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:11.645896 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59e87e-8716-4439-a0d9-1334a6dde2c9 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:11.646204 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59e87e-8716-4439-a0d9-1334a6dde2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:11.646443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59e87e-8716-4439-a0d9-1334a6dde2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:11.658097 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:11.663496 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa95e93-c442-4e27-8670-d49f39c6deb9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:11.665992 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4e59e87e-8716-4439-a0d9-1334a6dde2c9 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:11.668058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4e59e87e-8716-4439-a0d9-1334a6dde2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:11.705296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50092332-c8b3-4403-9c0c-f9e039e4a874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:11.705938 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50092332-c8b3-4403-9c0c-f9e039e4a874 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:11.706220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50092332-c8b3-4403-9c0c-f9e039e4a874 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:11.706488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50092332-c8b3-4403-9c0c-f9e039e4a874 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:11.714891 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50092332-c8b3-4403-9c0c-f9e039e4a874 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:11.716347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50092332-c8b3-4403-9c0c-f9e039e4a874 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:11.729930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-50092332-c8b3-4403-9c0c-f9e039e4a874 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:02:11.729930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:11.729930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:11.729930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:11.729930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:11.729930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:11.729930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:11.729930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:11.729930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:11.729930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:11.729930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:11.729930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:11.729930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:02:11.729930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:02:11.729930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:02:11.729930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:02:11.731214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:11.731214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:11.731214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:02:11.731214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:02:11.731214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:02:11.731214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:02:11.731214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:11.731214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:11.731214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:11.731214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:11.731214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:11.731214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:11.731214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:11.731214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:11.731214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:11.731214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:11.731214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:11.731214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:11.732341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:11.732341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:11.732341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:02:11.732341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:02:11.732341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:02:11.732341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:02:11.732341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:02:11.732341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:02:11.732341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:02:11.732341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:11.732341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 517/1037] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Sep 6 16:02:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:11.737759 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50092332-c8b3-4403-9c0c-f9e039e4a874 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:11.751574 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:11.763401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 521/1038] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:02:11.913121 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50092332-c8b3-4403-9c0c-f9e039e4a874 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:11.976043 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59e87e-8716-4439-a0d9-1334a6dde2c9 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:11.977728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430fec3-2c65-4c47-bb81-31e3f362d13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:11.980956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430fec3-2c65-4c47-bb81-31e3f362d13f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:11.981750 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430fec3-2c65-4c47-bb81-31e3f362d13f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:11.982508 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430fec3-2c65-4c47-bb81-31e3f362d13f None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:11.983926 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430fec3-2c65-4c47-bb81-31e3f362d13f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:11.984689 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430fec3-2c65-4c47-bb81-31e3f362d13f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:11.989621 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430fec3-2c65-4c47-bb81-31e3f362d13f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:12.039702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 518/1039] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Sep 6 16:02:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:12.044975 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430fec3-2c65-4c47-bb81-31e3f362d13f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:12.068958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:12.090212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59e87e-8716-4439-a0d9-1334a6dde2c9 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:12.128373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cde145-9c5d-4484-b27a-37e991d6e017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:12.129010 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cde145-9c5d-4484-b27a-37e991d6e017 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:12.129323 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cde145-9c5d-4484-b27a-37e991d6e017 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:12.130092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cde145-9c5d-4484-b27a-37e991d6e017 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:12.131304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cde145-9c5d-4484-b27a-37e991d6e017 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:12.132776 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cde145-9c5d-4484-b27a-37e991d6e017 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:12.134426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 522/1040] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:12.135336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cde145-9c5d-4484-b27a-37e991d6e017 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:12.166286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430fec3-2c65-4c47-bb81-31e3f362d13f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:12.214681 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e3150-4131-4e9b-a601-eef13e9710c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:12.215531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e3150-4131-4e9b-a601-eef13e9710c4 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:12.219012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e3150-4131-4e9b-a601-eef13e9710c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:12.219012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e3150-4131-4e9b-a601-eef13e9710c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:12.219843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:12.279931 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cde145-9c5d-4484-b27a-37e991d6e017 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:12.306828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e3150-4131-4e9b-a601-eef13e9710c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:12.308549 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e3150-4131-4e9b-a601-eef13e9710c4 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:12.320681 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2b8b2-73f4-4a15-8eef-1d42f116f775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:12.321337 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2b8b2-73f4-4a15-8eef-1d42f116f775 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:12.321613 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2b8b2-73f4-4a15-8eef-1d42f116f775 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:12.321883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2b8b2-73f4-4a15-8eef-1d42f116f775 None admin] PATH_INFO: `/v3/services/image` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:12.322502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2b8b2-73f4-4a15-8eef-1d42f116f775 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:12.323924 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2b8b2-73f4-4a15-8eef-1d42f116f775 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:12.328689 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:12.334156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-31b2b8b2-73f4-4a15-8eef-1d42f116f775 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 16:02:12.334156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:12.334156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:02:12.334156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:02:12.334156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:02:12.334156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:02:12.334156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:02:12.334156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:02:12.334156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 16:02:12.334156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:12.334156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:02:12.334156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:12.334156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:12.334156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:12.334156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:12.334156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:12.334156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:12.335887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:12.335887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:12.335887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:12.335887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:12.335887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:12.335887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:12.335887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:02:12.335887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:02:12.335887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:02:12.335887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:02:12.335887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:12.335887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:12.335887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:12.335887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:12.335887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:12.335887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:12.336929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:12.336929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:12.336929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:12.336929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:12.336929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:12.336929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:12.336929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:12.336929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:12.336929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:12.336929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:12.336929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:02:12.336929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:02:12.336929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 16:02:12.336929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:12.336929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 523/1041] 10.222.0.26 () {56 vars in 1105 bytes} [Tue Sep 6 16:02:12 2022] GET /identity/v3/services/image => generated 86 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:12.337837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2b8b2-73f4-4a15-8eef-1d42f116f775 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:12.436902 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 519/1042] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:12 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 16:02:12.437778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e3150-4131-4e9b-a601-eef13e9710c4 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:12.466288 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2b8b2-73f4-4a15-8eef-1d42f116f775 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:12.498044 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:12.509056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1dff15-4a1c-4143-88a7-5fba2b7895b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:12.509985 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1dff15-4a1c-4143-88a7-5fba2b7895b8 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:12.510444 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1dff15-4a1c-4143-88a7-5fba2b7895b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:12.510753 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1dff15-4a1c-4143-88a7-5fba2b7895b8 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:12.511430 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1dff15-4a1c-4143-88a7-5fba2b7895b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:12.512875 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1dff15-4a1c-4143-88a7-5fba2b7895b8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:12.540221 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 524/1043] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 6 16:02:12 2022] POST /identity/v3/users => generated 285 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:12.541859 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1dff15-4a1c-4143-88a7-5fba2b7895b8 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:12.567941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e3150-4131-4e9b-a601-eef13e9710c4 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:12.601376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:12.625954 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f1c2d-0866-4e24-ab6b-e1a91730abf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:12.626654 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f1c2d-0866-4e24-ab6b-e1a91730abf6 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:12.627005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f1c2d-0866-4e24-ab6b-e1a91730abf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:12.627288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f1c2d-0866-4e24-ab6b-e1a91730abf6 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:12.628021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f1c2d-0866-4e24-ab6b-e1a91730abf6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:12.628335 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f1c2d-0866-4e24-ab6b-e1a91730abf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:12.630469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f1c2d-0866-4e24-ab6b-e1a91730abf6 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:12.636597 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 520/1044] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:12.643102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f1c2d-0866-4e24-ab6b-e1a91730abf6 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:12.744885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1dff15-4a1c-4143-88a7-5fba2b7895b8 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:12.804253 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232db219-59e3-4ff1-b27d-3f58bbee0e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:12.804813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232db219-59e3-4ff1-b27d-3f58bbee0e05 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:12.805093 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232db219-59e3-4ff1-b27d-3f58bbee0e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:12.806782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232db219-59e3-4ff1-b27d-3f58bbee0e05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:12.817788 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f1c2d-0866-4e24-ab6b-e1a91730abf6 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:12.824842 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232db219-59e3-4ff1-b27d-3f58bbee0e05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:12.827172 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232db219-59e3-4ff1-b27d-3f58bbee0e05 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:12.854202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-232db219-59e3-4ff1-b27d-3f58bbee0e05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:02:12.854202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:12.854202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:12.854202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:12.854202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:12.854202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:12.854202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:12.854202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:12.854202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:12.854202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:12.854202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:12.854202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:12.854202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:02:12.854202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:02:12.854202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:02:12.854202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:02:12.855314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:12.855314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:12.855314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:12.855314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:12.855314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:12.855314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:12.855314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:12.855314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:12.855314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:12.855314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:12.855314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:12.855314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:12.855314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:12.855314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:12.855314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:12.855314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:12.855314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:02:12.856198 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:02:12.856198 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:02:12.856198 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:12.856925 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 525/1045] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:12.861799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232db219-59e3-4ff1-b27d-3f58bbee0e05 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:12.864086 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248d254-d162-4604-9b35-bbc10fb73780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:12.865152 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248d254-d162-4604-9b35-bbc10fb73780 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:12.865664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248d254-d162-4604-9b35-bbc10fb73780 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:12.866159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248d254-d162-4604-9b35-bbc10fb73780 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:12.867365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248d254-d162-4604-9b35-bbc10fb73780 None admin] RBAC: Adding query filter params (type=image) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:12.867901 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248d254-d162-4604-9b35-bbc10fb73780 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:12.870110 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248d254-d162-4604-9b35-bbc10fb73780 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:12.885272 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 521/1046] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:12.886867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248d254-d162-4604-9b35-bbc10fb73780 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:12.908505 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:12.963461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:12.990234 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232db219-59e3-4ff1-b27d-3f58bbee0e05 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:13.031901 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248d254-d162-4604-9b35-bbc10fb73780 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:13.046802 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673fb97-c3f9-445b-a638-b2c6a2d46827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:13.049110 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673fb97-c3f9-445b-a638-b2c6a2d46827 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:13.050164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673fb97-c3f9-445b-a638-b2c6a2d46827 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:13.050926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673fb97-c3f9-445b-a638-b2c6a2d46827 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:13.052313 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673fb97-c3f9-445b-a638-b2c6a2d46827 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:13.052713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673fb97-c3f9-445b-a638-b2c6a2d46827 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:13.054161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673fb97-c3f9-445b-a638-b2c6a2d46827 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:13.071129 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 526/1047] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:13.073388 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673fb97-c3f9-445b-a638-b2c6a2d46827 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:13.080848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c53d7-3773-4697-87a0-8c30e1645a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:13.081470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c53d7-3773-4697-87a0-8c30e1645a59 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:13.081751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c53d7-3773-4697-87a0-8c30e1645a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:13.082019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c53d7-3773-4697-87a0-8c30e1645a59 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:13.083266 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c53d7-3773-4697-87a0-8c30e1645a59 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:13.084641 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c53d7-3773-4697-87a0-8c30e1645a59 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:13.114535 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 522/1048] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:02:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:02:13.209498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:13.219443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673fb97-c3f9-445b-a638-b2c6a2d46827 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:13.292537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e57ee-21b1-4644-9222-31e87e89e482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:13.293222 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e57ee-21b1-4644-9222-31e87e89e482 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:13.293479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e57ee-21b1-4644-9222-31e87e89e482 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:13.293686 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e57ee-21b1-4644-9222-31e87e89e482 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:13.302336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998e57ee-21b1-4644-9222-31e87e89e482 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:13.303857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998e57ee-21b1-4644-9222-31e87e89e482 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:13.317996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-998e57ee-21b1-4644-9222-31e87e89e482 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:02:13.317996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:13.317996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:13.317996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:13.317996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:13.317996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:13.317996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:13.317996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:13.317996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:13.317996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:13.317996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:13.317996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:13.317996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:02:13.317996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:02:13.317996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:02:13.319210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:02:13.319210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:13.319210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:13.319210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:13.319210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:13.319210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:13.319210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:13.319210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:13.319210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:13.319210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:13.319210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:13.319210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:13.319210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:13.319210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:13.319210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:13.319210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:13.319210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:13.320190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:02:13.320190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:02:13.320190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:02:13.320190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:02:13.320190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:02:13.320190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:02:13.320190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:02:13.320190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:13.320190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 527/1049] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:02:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:13.325184 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c53d7-3773-4697-87a0-8c30e1645a59 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:13.452191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:13.456692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c53d7-3773-4697-87a0-8c30e1645a59 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:13.502022 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f66445-45d2-47f8-8de8-0af99e348c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:13.502641 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f66445-45d2-47f8-8de8-0af99e348c1f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:13.502953 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f66445-45d2-47f8-8de8-0af99e348c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:13.503276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f66445-45d2-47f8-8de8-0af99e348c1f None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:13.503976 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f66445-45d2-47f8-8de8-0af99e348c1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:13.504344 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f66445-45d2-47f8-8de8-0af99e348c1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:13.506499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f66445-45d2-47f8-8de8-0af99e348c1f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:13.521868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 523/1050] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:02:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:13.528297 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e57ee-21b1-4644-9222-31e87e89e482 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:13.680152 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e57ee-21b1-4644-9222-31e87e89e482 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:13.721959 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccc919-776b-498f-8a1b-6c6cac938603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:13.722646 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccc919-776b-498f-8a1b-6c6cac938603 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:13.723079 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccc919-776b-498f-8a1b-6c6cac938603 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:13.723900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccc919-776b-498f-8a1b-6c6cac938603 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:13.724892 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccc919-776b-498f-8a1b-6c6cac938603 None admin] RBAC: Adding query filter params (role.id=91f753ea35ce4f5a953d0a503481da1a, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=6afd4f6ffdae4b16aa5ba757e68c18c4) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:13.725254 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccc919-776b-498f-8a1b-6c6cac938603 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:13.726923 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccc919-776b-498f-8a1b-6c6cac938603 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:13.739138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 528/1051] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:13 2022] GET /identity/v3/role_assignments?user.id=6afd4f6ffdae4b16aa5ba757e68c18c4&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=91f753ea35ce4f5a953d0a503481da1a => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:14.650089 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de53a0-8f2e-4cc6-b176-c3f22d504f62 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:14.650694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de53a0-8f2e-4cc6-b176-c3f22d504f62 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:14.650694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de53a0-8f2e-4cc6-b176-c3f22d504f62 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:14.652044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 524/1052] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:14.657360 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7048e1-0816-45d2-b6e3-aaa92d57d71a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:14.657877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7048e1-0816-45d2-b6e3-aaa92d57d71a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:14.657954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7048e1-0816-45d2-b6e3-aaa92d57d71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:14.677997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ef7048e1-0816-45d2-b6e3-aaa92d57d71a None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:14.679259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ef7048e1-0816-45d2-b6e3-aaa92d57d71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:14.692996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:14.764280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 529/1053] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:15.041485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de53a0-8f2e-4cc6-b176-c3f22d504f62 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:15.151086 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de53a0-8f2e-4cc6-b176-c3f22d504f62 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:15.187415 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146e7fe-5559-4cee-8e02-f28892a078bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:15.188058 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146e7fe-5559-4cee-8e02-f28892a078bd None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:15.188337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146e7fe-5559-4cee-8e02-f28892a078bd None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:15.188610 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146e7fe-5559-4cee-8e02-f28892a078bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:15.196622 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35aa88-233e-4c60-90ed-db878b55fdad None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:15.196906 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35aa88-233e-4c60-90ed-db878b55fdad None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:15.197181 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35aa88-233e-4c60-90ed-db878b55fdad None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:15.198548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 530/1054] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:15.202755 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146e7fe-5559-4cee-8e02-f28892a078bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:15.204051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299470b9-2182-40bd-8a96-489e14b0df48 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:15.204334 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299470b9-2182-40bd-8a96-489e14b0df48 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:15.204605 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299470b9-2182-40bd-8a96-489e14b0df48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:15.208584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-c146e7fe-5559-4cee-8e02-f28892a078bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:15.208584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:15.208584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:15.208584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:15.208584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:15.208584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:15.208584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:15.208584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:15.208584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:15.208584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:15.208584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:15.208584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:15.208584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:15.208584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:15.208584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:15.208584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:15.209516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:15.209516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:15.209516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:15.209516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:15.209516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:15.209516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:15.209516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:15.209516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:15.209516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:15.209516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:15.209516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:15.209516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:15.209516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:15.209516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:15.209516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:15.209516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:15.210374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:15.210374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:15.210374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:15.210374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:15.210374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:15.210374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:15.210374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:15.210374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:15.210374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:15.210374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:15.210374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:15.210374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:15.210374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 525/1055] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:02:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:15.215666 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146e7fe-5559-4cee-8e02-f28892a078bd None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:15.227788 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-299470b9-2182-40bd-8a96-489e14b0df48 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:15.229069 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-299470b9-2182-40bd-8a96-489e14b0df48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:15.239108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:15.305570 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 531/1056] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:02:15.336179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146e7fe-5559-4cee-8e02-f28892a078bd None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:15.378822 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa2070-c061-4205-b609-13eeefe2233a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:15.380174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa2070-c061-4205-b609-13eeefe2233a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:15.380530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa2070-c061-4205-b609-13eeefe2233a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:15.381004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa2070-c061-4205-b609-13eeefe2233a None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:15.381678 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa2070-c061-4205-b609-13eeefe2233a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:15.382702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa2070-c061-4205-b609-13eeefe2233a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:15.383458 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa2070-c061-4205-b609-13eeefe2233a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:15.395520 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 526/1057] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:15.399140 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299470b9-2182-40bd-8a96-489e14b0df48 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:15.480626 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:15.517770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299470b9-2182-40bd-8a96-489e14b0df48 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:15.557499 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee131e-3c52-4803-bbb6-f85c55cf395c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:15.558046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee131e-3c52-4803-bbb6-f85c55cf395c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:15.558317 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee131e-3c52-4803-bbb6-f85c55cf395c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:15.558582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee131e-3c52-4803-bbb6-f85c55cf395c None admin] PATH_INFO: `/v3/users/5f141a2abdbf417fa5be5a15f5db40fd` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:15.583632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee131e-3c52-4803-bbb6-f85c55cf395c None admin] RBAC: Authorizing `identity:get_user(user_id=5f141a2abdbf417fa5be5a15f5db40fd)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:15.586245 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee131e-3c52-4803-bbb6-f85c55cf395c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:15.594283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 532/1058] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:15 2022] GET /identity/v3/users/5f141a2abdbf417fa5be5a15f5db40fd => generated 285 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:15.595398 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa2070-c061-4205-b609-13eeefe2233a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:15.609349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee131e-3c52-4803-bbb6-f85c55cf395c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:15.695993 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:15.721569 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa2070-c061-4205-b609-13eeefe2233a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:15.748185 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee131e-3c52-4803-bbb6-f85c55cf395c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:15.766953 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c531c9-51f9-41e4-a686-2d3b391a1b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:15.767580 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c531c9-51f9-41e4-a686-2d3b391a1b9c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:15.767863 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c531c9-51f9-41e4-a686-2d3b391a1b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:15.768229 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c531c9-51f9-41e4-a686-2d3b391a1b9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:15.790196 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:15.796370 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38c22a-76cd-4ddf-b724-dd1e212c5d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:15.797045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38c22a-76cd-4ddf-b724-dd1e212c5d37 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:15.797503 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38c22a-76cd-4ddf-b724-dd1e212c5d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:15.797732 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38c22a-76cd-4ddf-b724-dd1e212c5d37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:15.855603 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c531c9-51f9-41e4-a686-2d3b391a1b9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:15.857103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c531c9-51f9-41e4-a686-2d3b391a1b9c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:15.883854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38c22a-76cd-4ddf-b724-dd1e212c5d37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:15.885473 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38c22a-76cd-4ddf-b724-dd1e212c5d37 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:15.970079 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:15.994852 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 527/1059] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:02:15.996182 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5b727-e736-4e9a-9894-91df4ac12a59 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:15.996463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5b727-e736-4e9a-9894-91df4ac12a59 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:15.996733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5b727-e736-4e9a-9894-91df4ac12a59 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:15.998208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 528/1060] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:16.001425 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5b727-e736-4e9a-9894-91df4ac12a59 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:16.023547 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 533/1061] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:02:16.025498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81220e71-6f9a-41e1-94b8-811ee554746a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:16.025498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81220e71-6f9a-41e1-94b8-811ee554746a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:16.025693 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81220e71-6f9a-41e1-94b8-811ee554746a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:16.051917 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-81220e71-6f9a-41e1-94b8-811ee554746a None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:16.054016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-81220e71-6f9a-41e1-94b8-811ee554746a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:16.087526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:16.137484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 534/1062] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:16.138648 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81220e71-6f9a-41e1-94b8-811ee554746a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:16.150579 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5b727-e736-4e9a-9894-91df4ac12a59 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:16.195147 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d67f2f-eef9-4991-bd14-3dfaf1ce8adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:16.196236 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d67f2f-eef9-4991-bd14-3dfaf1ce8adc None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:16.196707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d67f2f-eef9-4991-bd14-3dfaf1ce8adc None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:16.197169 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d67f2f-eef9-4991-bd14-3dfaf1ce8adc None admin] PATH_INFO: `/v3/projects/b0eb7b2efa6c4d7382a8ad7ec9363d61` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:16.199228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d67f2f-eef9-4991-bd14-3dfaf1ce8adc None admin] RBAC: Authorizing `identity:get_project(project_id=b0eb7b2efa6c4d7382a8ad7ec9363d61)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:16.200688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d67f2f-eef9-4991-bd14-3dfaf1ce8adc None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:16.202383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 529/1063] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:16 2022] GET /identity/v3/projects/b0eb7b2efa6c4d7382a8ad7ec9363d61 => generated 311 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:16.209157 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d67f2f-eef9-4991-bd14-3dfaf1ce8adc None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:16.283395 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81220e71-6f9a-41e1-94b8-811ee554746a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:16.287770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:16.326513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63178d-f8ec-4736-bf3b-db6cc114e759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:16.327459 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63178d-f8ec-4736-bf3b-db6cc114e759 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:16.327912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63178d-f8ec-4736-bf3b-db6cc114e759 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:16.328403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63178d-f8ec-4736-bf3b-db6cc114e759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:16.344448 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63178d-f8ec-4736-bf3b-db6cc114e759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:16.346297 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63178d-f8ec-4736-bf3b-db6cc114e759 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:16.356681 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d67f2f-eef9-4991-bd14-3dfaf1ce8adc None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:16.369501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-1f63178d-f8ec-4736-bf3b-db6cc114e759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:02:16.369501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:16.369501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:16.369501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:16.369501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:16.369501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:16.369501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:16.369501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:16.369501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:16.369501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:16.369501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:16.369501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:16.369501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:02:16.369501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:02:16.369501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:02:16.369501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:02:16.370910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:16.370910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:16.370910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:16.370910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:16.370910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:16.370910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:16.370910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:16.370910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:16.370910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:16.370910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:16.370910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:16.370910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:16.370910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:16.370910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:16.370910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:16.370910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:16.370910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:02:16.372115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:02:16.372115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:02:16.372115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:16.372115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 535/1064] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:16.376867 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63178d-f8ec-4736-bf3b-db6cc114e759 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:16.395882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:16.406191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3df3d-71f7-4a1b-bdad-5bb847ae6544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:16.406847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3df3d-71f7-4a1b-bdad-5bb847ae6544 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:16.407124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3df3d-71f7-4a1b-bdad-5bb847ae6544 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:16.407392 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3df3d-71f7-4a1b-bdad-5bb847ae6544 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:16.408118 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3df3d-71f7-4a1b-bdad-5bb847ae6544 None admin] RBAC: Adding query filter params (role.id=ba05677650e046a48d5f19440d48fbf2, scope.project.id=b0eb7b2efa6c4d7382a8ad7ec9363d61, user.id=5f141a2abdbf417fa5be5a15f5db40fd) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:16.408427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3df3d-71f7-4a1b-bdad-5bb847ae6544 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:16.409887 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3df3d-71f7-4a1b-bdad-5bb847ae6544 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:16.417954 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 530/1065] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:16 2022] GET /identity/v3/role_assignments?user.id=5f141a2abdbf417fa5be5a15f5db40fd&scope.project.id=b0eb7b2efa6c4d7382a8ad7ec9363d61&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:16.464955 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3df3d-71f7-4a1b-bdad-5bb847ae6544 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:16.510270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63178d-f8ec-4736-bf3b-db6cc114e759 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:16.575052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31f213-5763-419c-89c0-d24fc07120c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:16.575579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31f213-5763-419c-89c0-d24fc07120c6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:16.575861 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31f213-5763-419c-89c0-d24fc07120c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:16.576137 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31f213-5763-419c-89c0-d24fc07120c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:16.576785 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31f213-5763-419c-89c0-d24fc07120c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:16.577097 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31f213-5763-419c-89c0-d24fc07120c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:16.578483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31f213-5763-419c-89c0-d24fc07120c6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:16.601039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 536/1066] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:16.616444 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3df3d-71f7-4a1b-bdad-5bb847ae6544 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:16.643333 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:16.677182 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabee93-09af-4e45-8dcf-4b9b72b50fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:16.677727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabee93-09af-4e45-8dcf-4b9b72b50fce None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:16.678006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabee93-09af-4e45-8dcf-4b9b72b50fce None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:16.678270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabee93-09af-4e45-8dcf-4b9b72b50fce None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:16.693471 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabee93-09af-4e45-8dcf-4b9b72b50fce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:16.698560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-5cabee93-09af-4e45-8dcf-4b9b72b50fce None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:02:16.698560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:16.698560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:16.698560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:16.698560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:16.698560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:16.698560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:16.698560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:16.698560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:16.698560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:16.698560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:16.698560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:16.698560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:16.698560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:16.698560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:16.698560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:16.700033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:16.700033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:16.700033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:16.700033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:16.700033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:16.700033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:16.700033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:16.700033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:16.700033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:16.700033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:16.700033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:16.700033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:16.700033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:16.700033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:16.700033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:16.700033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:16.700909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:16.700909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:16.700909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:16.700909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:16.700909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:16.700909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:16.700909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:16.700909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:16.700909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:16.700909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:16.700909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:02:16.700909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:16.700909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 531/1067] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Sep 6 16:02:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:02:16.706814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31f213-5763-419c-89c0-d24fc07120c6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:16.813225 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:16.847556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31f213-5763-419c-89c0-d24fc07120c6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:16.903840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffe476-4fb9-4363-b6b8-54179f92b107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:16.904076 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffe476-4fb9-4363-b6b8-54179f92b107 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:16.905799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffe476-4fb9-4363-b6b8-54179f92b107 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:16.905799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffe476-4fb9-4363-b6b8-54179f92b107 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:16.906104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffe476-4fb9-4363-b6b8-54179f92b107 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:16.906674 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffe476-4fb9-4363-b6b8-54179f92b107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:16.909052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffe476-4fb9-4363-b6b8-54179f92b107 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:16.924277 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 537/1068] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Sep 6 16:02:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:16.930759 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabee93-09af-4e45-8dcf-4b9b72b50fce None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:17.050079 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabee93-09af-4e45-8dcf-4b9b72b50fce None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:17.108509 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5cda7-27fb-461f-9620-bf4db4a27539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:17.109537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5cda7-27fb-461f-9620-bf4db4a27539 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:17.110020 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5cda7-27fb-461f-9620-bf4db4a27539 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:17.110509 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5cda7-27fb-461f-9620-bf4db4a27539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:17.128645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5cda7-27fb-461f-9620-bf4db4a27539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:17.130694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5cda7-27fb-461f-9620-bf4db4a27539 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:17.146247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-6eb5cda7-27fb-461f-9620-bf4db4a27539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:02:17.146247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:17.146247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:17.146247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:17.146247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:17.146247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:17.146247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:17.146247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:17.146247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:17.146247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:17.146247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:17.146247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:17.146247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:02:17.146247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:02:17.146247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:02:17.146247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:02:17.147528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:17.147528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:17.147528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:17.147528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:17.147528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:17.147528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:17.147528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:17.147528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:17.147528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:17.147528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:17.147528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:17.147528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:17.147528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:17.147528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:17.147528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:17.147528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:17.147528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:02:17.148985 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:02:17.148985 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:02:17.148985 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:17.148985 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 532/1069] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:17.153324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffe476-4fb9-4363-b6b8-54179f92b107 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:17.262472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffe476-4fb9-4363-b6b8-54179f92b107 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:17.299723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42073c3d-64dc-4ade-807d-6a413ea7448c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:17.300328 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42073c3d-64dc-4ade-807d-6a413ea7448c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:17.300601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42073c3d-64dc-4ade-807d-6a413ea7448c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:17.300873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42073c3d-64dc-4ade-807d-6a413ea7448c None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:17.301869 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42073c3d-64dc-4ade-807d-6a413ea7448c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:17.302170 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42073c3d-64dc-4ade-807d-6a413ea7448c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:17.303579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42073c3d-64dc-4ade-807d-6a413ea7448c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:17.306204 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:17.317130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 538/1070] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:17.323123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5cda7-27fb-461f-9620-bf4db4a27539 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:17.353311 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:17.431844 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5cda7-27fb-461f-9620-bf4db4a27539 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:17.475424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545c1fc-1cbb-4993-b614-115556500487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:17.476316 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545c1fc-1cbb-4993-b614-115556500487 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:17.476779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545c1fc-1cbb-4993-b614-115556500487 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:17.477133 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545c1fc-1cbb-4993-b614-115556500487 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:17.485551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545c1fc-1cbb-4993-b614-115556500487 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:17.487038 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545c1fc-1cbb-4993-b614-115556500487 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:17.499160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-a545c1fc-1cbb-4993-b614-115556500487 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:02:17.499160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:17.499160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:17.499160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:17.499160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:17.499160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:17.499160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:17.499160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:17.499160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:17.499160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:17.499160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:17.499160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:17.499160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:02:17.499160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:02:17.499160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:02:17.499160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:02:17.500817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:17.500817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:17.500817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:02:17.500817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:02:17.500817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:02:17.500817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:02:17.500817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:17.500817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:17.500817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:17.500817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:17.500817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:17.500817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:17.500817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:17.500817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:17.500817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:17.500817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:17.500817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:17.500817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:17.502293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:17.502293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:17.502293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:02:17.502293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:02:17.502293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:02:17.502293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:02:17.502293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:02:17.502293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:02:17.502293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:02:17.502293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:17.502293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 533/1071] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Sep 6 16:02:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:17.507910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42073c3d-64dc-4ade-807d-6a413ea7448c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:17.652840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42073c3d-64dc-4ade-807d-6a413ea7448c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:17.697510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbde852-33f7-47e3-ae16-d79ad6ec48e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:17.698112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbde852-33f7-47e3-ae16-d79ad6ec48e4 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:17.698387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbde852-33f7-47e3-ae16-d79ad6ec48e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:17.698655 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbde852-33f7-47e3-ae16-d79ad6ec48e4 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:17.699394 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbde852-33f7-47e3-ae16-d79ad6ec48e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:17.699694 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbde852-33f7-47e3-ae16-d79ad6ec48e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:17.701097 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbde852-33f7-47e3-ae16-d79ad6ec48e4 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:17.719592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 539/1072] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Sep 6 16:02:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:17.726239 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545c1fc-1cbb-4993-b614-115556500487 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:17.829733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:17.845808 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545c1fc-1cbb-4993-b614-115556500487 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:17.886373 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1783827-f5ee-4166-ae5d-809f2b949679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:17.886975 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1783827-f5ee-4166-ae5d-809f2b949679 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:17.887256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1783827-f5ee-4166-ae5d-809f2b949679 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:17.887527 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1783827-f5ee-4166-ae5d-809f2b949679 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:17.899839 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1783827-f5ee-4166-ae5d-809f2b949679 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:17.901462 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1783827-f5ee-4166-ae5d-809f2b949679 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:17.918588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-c1783827-f5ee-4166-ae5d-809f2b949679 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:02:17.918588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:17.918588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:17.918588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:17.918588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:17.918588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:17.918588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:17.918588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:17.918588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:17.918588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:17.918588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:17.918588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:17.918588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:02:17.918588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:02:17.918588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:02:17.918588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:02:17.919780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:17.919780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:17.919780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:17.919780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:17.919780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:17.919780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:17.919780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:17.919780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:17.919780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:17.919780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:17.919780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:17.919780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:17.919780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:17.919780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:17.919780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:17.919780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:17.919780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:02:17.921607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:02:17.921607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:02:17.921607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:17.921607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 534/1073] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:17.926878 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbde852-33f7-47e3-ae16-d79ad6ec48e4 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:17.992041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:18.066056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbde852-33f7-47e3-ae16-d79ad6ec48e4 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:18.125295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e8efa-0ff9-4f15-ab6b-7e71686b12e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:18.126345 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e8efa-0ff9-4f15-ab6b-7e71686b12e1 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:18.126905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e8efa-0ff9-4f15-ab6b-7e71686b12e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:18.127393 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e8efa-0ff9-4f15-ab6b-7e71686b12e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:18.128545 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e8efa-0ff9-4f15-ab6b-7e71686b12e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:18.129125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e8efa-0ff9-4f15-ab6b-7e71686b12e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:18.131689 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e8efa-0ff9-4f15-ab6b-7e71686b12e1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:18.144398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 540/1074] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:18.150701 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1783827-f5ee-4166-ae5d-809f2b949679 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:18.271221 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1783827-f5ee-4166-ae5d-809f2b949679 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:18.308501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce1274-645c-412d-96a1-800a94e9d3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:18.309130 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce1274-645c-412d-96a1-800a94e9d3ee None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:18.309407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce1274-645c-412d-96a1-800a94e9d3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:18.309678 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce1274-645c-412d-96a1-800a94e9d3ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:18.319562 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce1274-645c-412d-96a1-800a94e9d3ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:18.321125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce1274-645c-412d-96a1-800a94e9d3ee None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:18.323213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:18.334779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-a4ce1274-645c-412d-96a1-800a94e9d3ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:02:18.334779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:18.334779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:18.334779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:18.334779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:18.334779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:18.334779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:18.334779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:18.334779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:18.334779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:18.334779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:18.334779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:18.334779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:02:18.334779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:02:18.334779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:02:18.336042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:02:18.336042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:18.336042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:18.336042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:18.336042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:18.336042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:18.336042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:18.336042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:18.336042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:18.336042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:18.336042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:18.336042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:18.336042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:18.336042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:18.336042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:18.336042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:18.336042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:18.337165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:02:18.337165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:02:18.337165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:02:18.337165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:02:18.337165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:02:18.337165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:02:18.337165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:02:18.337165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:18.337165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 535/1075] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:02:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:18.342513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e8efa-0ff9-4f15-ab6b-7e71686b12e1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:18.452340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e8efa-0ff9-4f15-ab6b-7e71686b12e1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:18.458626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe16aa6-f08f-4d60-a511-aaf1c9271018 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:18.458966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe16aa6-f08f-4d60-a511-aaf1c9271018 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:18.459247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe16aa6-f08f-4d60-a511-aaf1c9271018 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:18.460602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 536/1076] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:18.466233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b429311-26b3-4a50-a474-9d022f175b4b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:18.466528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b429311-26b3-4a50-a474-9d022f175b4b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:18.466852 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b429311-26b3-4a50-a474-9d022f175b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:18.468101 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:18.487570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-4b429311-26b3-4a50-a474-9d022f175b4b None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:18.489451 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4b429311-26b3-4a50-a474-9d022f175b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:18.491394 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b624bd-9f07-4fa1-b532-504f76d5756a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:18.492009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b624bd-9f07-4fa1-b532-504f76d5756a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:18.492289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b624bd-9f07-4fa1-b532-504f76d5756a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:18.492570 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b624bd-9f07-4fa1-b532-504f76d5756a None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:18.493270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b624bd-9f07-4fa1-b532-504f76d5756a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:18.493579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b624bd-9f07-4fa1-b532-504f76d5756a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:18.495041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b624bd-9f07-4fa1-b532-504f76d5756a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:18.508706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 541/1077] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:02:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:18.518193 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b624bd-9f07-4fa1-b532-504f76d5756a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:18.571394 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 537/1078] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:02:18.630628 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b624bd-9f07-4fa1-b532-504f76d5756a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:18.666249 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164923c-7f6d-4ac9-bb31-88a39994dbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:18.666935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164923c-7f6d-4ac9-bb31-88a39994dbc3 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:18.667212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164923c-7f6d-4ac9-bb31-88a39994dbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:18.667485 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164923c-7f6d-4ac9-bb31-88a39994dbc3 None admin] PATH_INFO: `/v3/projects/0138d62d5879495499a17e20310d4f61/users/6afd4f6ffdae4b16aa5ba757e68c18c4/roles/91f753ea35ce4f5a953d0a503481da1a` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:18.669683 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164923c-7f6d-4ac9-bb31-88a39994dbc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=0138d62d5879495499a17e20310d4f61, user_id=6afd4f6ffdae4b16aa5ba757e68c18c4, role_id=91f753ea35ce4f5a953d0a503481da1a)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:18.671165 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164923c-7f6d-4ac9-bb31-88a39994dbc3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:18.686077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 542/1079] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:02:18 2022] PUT /identity/v3/projects/0138d62d5879495499a17e20310d4f61/users/6afd4f6ffdae4b16aa5ba757e68c18c4/roles/91f753ea35ce4f5a953d0a503481da1a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:02:18.854494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b429311-26b3-4a50-a474-9d022f175b4b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:18.965518 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b429311-26b3-4a50-a474-9d022f175b4b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:18.994903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:19.003935 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d8beb-224f-475d-83eb-5f175a129166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:19.005094 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d8beb-224f-475d-83eb-5f175a129166 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:19.005492 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d8beb-224f-475d-83eb-5f175a129166 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:19.005853 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d8beb-224f-475d-83eb-5f175a129166 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:19.017845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d8beb-224f-475d-83eb-5f175a129166 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:19.023554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-592d8beb-224f-475d-83eb-5f175a129166 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:19.023554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:19.023554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:19.023554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:19.023554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:19.023554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:19.023554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:19.023554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:19.023554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:19.023554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:19.023554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:19.023554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:19.023554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:19.023554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:19.023554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:19.023554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:19.024770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:19.024770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:19.024770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:19.024770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:19.024770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:19.024770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:19.024770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:19.024770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:19.024770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:19.024770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:19.024770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:19.024770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:19.024770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:19.024770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:19.024770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:19.024770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:19.026200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:19.026200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:19.026200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:19.026200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:19.026200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:19.026200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:19.026200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:19.026200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:19.026200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:19.026200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:19.026200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:19.026200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:19.026200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 538/1080] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:02:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:19.030880 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164923c-7f6d-4ac9-bb31-88a39994dbc3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:19.070905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:19.142294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164923c-7f6d-4ac9-bb31-88a39994dbc3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:19.178873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de370384-89b9-420e-b754-ef786bd873e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:19.179476 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de370384-89b9-420e-b754-ef786bd873e1 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:19.179755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de370384-89b9-420e-b754-ef786bd873e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:19.180024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de370384-89b9-420e-b754-ef786bd873e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:19.180688 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de370384-89b9-420e-b754-ef786bd873e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:19.180987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de370384-89b9-420e-b754-ef786bd873e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:19.182350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de370384-89b9-420e-b754-ef786bd873e1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:19.191699 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 543/1081] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:19.198313 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d8beb-224f-475d-83eb-5f175a129166 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:19.309982 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d8beb-224f-475d-83eb-5f175a129166 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:19.346122 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f93237-ed0b-47a4-ada9-7013b515536d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:19.346801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f93237-ed0b-47a4-ada9-7013b515536d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:19.347086 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f93237-ed0b-47a4-ada9-7013b515536d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:19.347358 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f93237-ed0b-47a4-ada9-7013b515536d None admin] PATH_INFO: `/v3/users/5f141a2abdbf417fa5be5a15f5db40fd` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:19.349159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f93237-ed0b-47a4-ada9-7013b515536d None admin] RBAC: Authorizing `identity:get_user(user_id=5f141a2abdbf417fa5be5a15f5db40fd)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:19.350605 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f93237-ed0b-47a4-ada9-7013b515536d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:19.352267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 539/1082] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:19 2022] GET /identity/v3/users/5f141a2abdbf417fa5be5a15f5db40fd => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:19.358069 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de370384-89b9-420e-b754-ef786bd873e1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:19.463786 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:19.470847 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de370384-89b9-420e-b754-ef786bd873e1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:19.507294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d7d8e-23dc-4dfe-859b-e385662a0465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:19.507920 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d7d8e-23dc-4dfe-859b-e385662a0465 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:19.508200 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d7d8e-23dc-4dfe-859b-e385662a0465 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:19.508471 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d7d8e-23dc-4dfe-859b-e385662a0465 None admin] PATH_INFO: `/v3/projects/b0eb7b2efa6c4d7382a8ad7ec9363d61` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:19.509610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d7d8e-23dc-4dfe-859b-e385662a0465 None admin] RBAC: Authorizing `identity:get_project(project_id=b0eb7b2efa6c4d7382a8ad7ec9363d61)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:19.511132 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d7d8e-23dc-4dfe-859b-e385662a0465 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:19.512793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 544/1083] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:19 2022] GET /identity/v3/projects/b0eb7b2efa6c4d7382a8ad7ec9363d61 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:19.518848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f93237-ed0b-47a4-ada9-7013b515536d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:19.586586 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:19.640935 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f93237-ed0b-47a4-ada9-7013b515536d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:19.681717 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e36ef-3074-409d-867c-f8ab6e4f8ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:19.682399 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e36ef-3074-409d-867c-f8ab6e4f8ebf None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:19.682705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e36ef-3074-409d-867c-f8ab6e4f8ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:19.682984 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e36ef-3074-409d-867c-f8ab6e4f8ebf None admin] PATH_INFO: `/v3/projects/b0eb7b2efa6c4d7382a8ad7ec9363d61/users/5f141a2abdbf417fa5be5a15f5db40fd/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:19.684989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e36ef-3074-409d-867c-f8ab6e4f8ebf None admin] RBAC: Authorizing `identity:create_grant(project_id=b0eb7b2efa6c4d7382a8ad7ec9363d61, user_id=5f141a2abdbf417fa5be5a15f5db40fd, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:19.686474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e36ef-3074-409d-867c-f8ab6e4f8ebf None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:19.701661 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 540/1084] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:02:19 2022] PUT /identity/v3/projects/b0eb7b2efa6c4d7382a8ad7ec9363d61/users/5f141a2abdbf417fa5be5a15f5db40fd/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:02:20.680420 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e408dbd-07cd-4cf3-a918-19281844be68 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:20.681538 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e408dbd-07cd-4cf3-a918-19281844be68 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:20.681538 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e408dbd-07cd-4cf3-a918-19281844be68 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:20.682521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 545/1085] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:20.687842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d28f16-e1ee-4dc2-bfdb-6f1fda56a7cc None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:20.688297 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d28f16-e1ee-4dc2-bfdb-6f1fda56a7cc None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:20.688390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d28f16-e1ee-4dc2-bfdb-6f1fda56a7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:20.706586 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-56d28f16-e1ee-4dc2-bfdb-6f1fda56a7cc None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:20.708102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-56d28f16-e1ee-4dc2-bfdb-6f1fda56a7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:20.777680 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 541/1086] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:21.046761 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e408dbd-07cd-4cf3-a918-19281844be68 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:21.159103 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e408dbd-07cd-4cf3-a918-19281844be68 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:21.197070 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcf5f1-e1cd-4b9d-8911-ee1a85dc4c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:21.198152 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcf5f1-e1cd-4b9d-8911-ee1a85dc4c51 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:21.198653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcf5f1-e1cd-4b9d-8911-ee1a85dc4c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:21.199073 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcf5f1-e1cd-4b9d-8911-ee1a85dc4c51 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:21.217928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcf5f1-e1cd-4b9d-8911-ee1a85dc4c51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:21.228539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-3fbcf5f1-e1cd-4b9d-8911-ee1a85dc4c51 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:02:21.228539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:21.228539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:21.228539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:21.228539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:21.228539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:21.228539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:21.228539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:21.228539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:21.228539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:21.228539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:21.228539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:21.228539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:21.228539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:21.228539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:21.228539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:21.229665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:21.229665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:21.229665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:21.229665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:21.229665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:21.229665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:21.229665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:21.229665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:21.229665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:21.229665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:21.229665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:21.229665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:21.229665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:21.229665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:21.229665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:21.229665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:21.230601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:21.230601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:21.230601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:21.230601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:21.230601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:21.230601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:21.230601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:21.230601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:21.230601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:21.230601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:21.230601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 16:02:21.230601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:21.231294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 546/1087] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Sep 6 16:02:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:21.236817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d28f16-e1ee-4dc2-bfdb-6f1fda56a7cc None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:21.291153 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:21.349218 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d28f16-e1ee-4dc2-bfdb-6f1fda56a7cc None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:21.386825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a71ea-6a10-4bd3-952a-df4733430f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:21.387452 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a71ea-6a10-4bd3-952a-df4733430f04 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:21.387726 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a71ea-6a10-4bd3-952a-df4733430f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:21.387992 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a71ea-6a10-4bd3-952a-df4733430f04 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:21.388670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a71ea-6a10-4bd3-952a-df4733430f04 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:21.388970 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a71ea-6a10-4bd3-952a-df4733430f04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:21.390371 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a71ea-6a10-4bd3-952a-df4733430f04 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:21.400339 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 542/1088] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Sep 6 16:02:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:21.406086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcf5f1-e1cd-4b9d-8911-ee1a85dc4c51 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:21.433439 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:21.519347 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcf5f1-e1cd-4b9d-8911-ee1a85dc4c51 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:21.555203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa4f55-c38e-4b5c-bbd1-ca61643f6b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:21.555820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa4f55-c38e-4b5c-bbd1-ca61643f6b83 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:21.556091 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa4f55-c38e-4b5c-bbd1-ca61643f6b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:21.556355 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa4f55-c38e-4b5c-bbd1-ca61643f6b83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:21.567238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa4f55-c38e-4b5c-bbd1-ca61643f6b83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:21.568766 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa4f55-c38e-4b5c-bbd1-ca61643f6b83 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:21.584441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-79fa4f55-c38e-4b5c-bbd1-ca61643f6b83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:02:21.584441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:21.584441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:21.584441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:21.584441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:21.584441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:21.584441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:21.584441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:21.584441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:21.584441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:21.584441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:21.584441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:21.584441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:02:21.584441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:02:21.584441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:02:21.584441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:02:21.585498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:21.585498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:21.585498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:21.585498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:21.585498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:21.585498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:21.585498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:21.585498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:21.585498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:21.585498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:21.585498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:21.585498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:21.585498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:21.585498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:21.585498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:21.585498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:21.585498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:02:21.586397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:02:21.586397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:02:21.586397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:21.586397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 547/1089] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:21.591459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a71ea-6a10-4bd3-952a-df4733430f04 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:21.691103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a71ea-6a10-4bd3-952a-df4733430f04 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:21.700613 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:21.719375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55724b0-1af6-46da-8acf-94df118758c8 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:21.719952 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55724b0-1af6-46da-8acf-94df118758c8 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:21.719952 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55724b0-1af6-46da-8acf-94df118758c8 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:21.721245 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 548/1090] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:02:21.726944 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb186a1-b08b-4fba-9930-6d09c0c362f2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:21.727091 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d251f3-2bc6-4d1b-81c4-333e0c3a9c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:21.727236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb186a1-b08b-4fba-9930-6d09c0c362f2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:21.727515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb186a1-b08b-4fba-9930-6d09c0c362f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:21.727629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d251f3-2bc6-4d1b-81c4-333e0c3a9c5e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:21.727873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d251f3-2bc6-4d1b-81c4-333e0c3a9c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:21.728140 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d251f3-2bc6-4d1b-81c4-333e0c3a9c5e None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:21.728781 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d251f3-2bc6-4d1b-81c4-333e0c3a9c5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:21.729080 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d251f3-2bc6-4d1b-81c4-333e0c3a9c5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:21.730460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d251f3-2bc6-4d1b-81c4-333e0c3a9c5e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:21.743503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 543/1091] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:21.747086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6eb186a1-b08b-4fba-9930-6d09c0c362f2 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:21.748303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6eb186a1-b08b-4fba-9930-6d09c0c362f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:21.749111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d251f3-2bc6-4d1b-81c4-333e0c3a9c5e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:21.818153 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 549/1092] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:02:21.850826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d251f3-2bc6-4d1b-81c4-333e0c3a9c5e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:21.885142 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459ffba-6f19-430d-90a3-f4d647970a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:21.885768 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459ffba-6f19-430d-90a3-f4d647970a10 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:21.886045 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459ffba-6f19-430d-90a3-f4d647970a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:21.886314 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459ffba-6f19-430d-90a3-f4d647970a10 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:21.893411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459ffba-6f19-430d-90a3-f4d647970a10 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:21.895066 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459ffba-6f19-430d-90a3-f4d647970a10 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:21.906547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-5459ffba-6f19-430d-90a3-f4d647970a10 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:02:21.906547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:21.906547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:21.906547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:21.906547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:21.906547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:21.906547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:21.906547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:21.906547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:21.906547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:21.906547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:21.906547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:21.906547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:02:21.906547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:02:21.906547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:02:21.906547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:02:21.912942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:21.912942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:21.912942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:02:21.912942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:02:21.912942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:02:21.912942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:02:21.912942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:21.912942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:21.912942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:21.912942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:21.912942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:21.912942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:21.912942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:21.912942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:21.912942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:21.912942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:21.912942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:21.912942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:21.913904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:21.913904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:21.913904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:02:21.913904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:02:21.913904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:02:21.913904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:02:21.913904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:02:21.913904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:02:21.913904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 16:02:21.913904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:21.913904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 544/1093] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Sep 6 16:02:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:02:21.914493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb186a1-b08b-4fba-9930-6d09c0c362f2 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:21.922577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:22.012770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb186a1-b08b-4fba-9930-6d09c0c362f2 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:22.046138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bb77b-aff6-4383-864b-c7e808b24046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:22.046760 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bb77b-aff6-4383-864b-c7e808b24046 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:22.047035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bb77b-aff6-4383-864b-c7e808b24046 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:22.047302 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bb77b-aff6-4383-864b-c7e808b24046 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:22.048008 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bb77b-aff6-4383-864b-c7e808b24046 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:22.048306 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bb77b-aff6-4383-864b-c7e808b24046 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:22.049696 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bb77b-aff6-4383-864b-c7e808b24046 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:22.067395 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 550/1094] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Sep 6 16:02:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:22.073168 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459ffba-6f19-430d-90a3-f4d647970a10 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:22.100025 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bb77b-aff6-4383-864b-c7e808b24046 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:22.102320 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:22.173460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:22.176979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459ffba-6f19-430d-90a3-f4d647970a10 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:22.213047 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78874b3-1aea-4b7b-8adb-fc3e48f133b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:22.213655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78874b3-1aea-4b7b-8adb-fc3e48f133b8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:22.213930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78874b3-1aea-4b7b-8adb-fc3e48f133b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:22.214200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78874b3-1aea-4b7b-8adb-fc3e48f133b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:22.214314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bb77b-aff6-4383-864b-c7e808b24046 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:22.249093 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e713db-0ec7-4ef0-b2ce-88500cf44efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:22.249718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e713db-0ec7-4ef0-b2ce-88500cf44efc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:22.249999 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e713db-0ec7-4ef0-b2ce-88500cf44efc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:22.250270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e713db-0ec7-4ef0-b2ce-88500cf44efc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:22.260929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e713db-0ec7-4ef0-b2ce-88500cf44efc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:22.265771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-e9e713db-0ec7-4ef0-b2ce-88500cf44efc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:22.265771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:22.265771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:22.265771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:22.265771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:22.265771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:22.265771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:22.265771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:22.265771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:22.265771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:22.265771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:22.265771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:22.265771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:22.265771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:22.265771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:22.265771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:22.266903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:22.266903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:22.266903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:22.266903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:22.266903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:22.266903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:22.266903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:22.266903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:22.266903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:22.266903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:22.266903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:22.266903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:22.266903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:22.266903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:22.266903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:22.266903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:22.267940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:22.267940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:22.267940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:22.267940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:22.267940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:22.267940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:22.267940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:22.267940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:22.267940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:22.267940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:22.267940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:22.267940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:22.273397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 551/1095] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:02:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:22.274125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e713db-0ec7-4ef0-b2ce-88500cf44efc None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:22.284405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78874b3-1aea-4b7b-8adb-fc3e48f133b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:22.286085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78874b3-1aea-4b7b-8adb-fc3e48f133b8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:22.313245 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:22.378595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e713db-0ec7-4ef0-b2ce-88500cf44efc None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:22.392273 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 545/1096] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 16:02:22.399555 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78874b3-1aea-4b7b-8adb-fc3e48f133b8 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:22.404365 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:22.416228 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee555d04-9443-4071-8d6c-cd735f434fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:22.416823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee555d04-9443-4071-8d6c-cd735f434fce None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:22.417097 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee555d04-9443-4071-8d6c-cd735f434fce None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:22.417367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee555d04-9443-4071-8d6c-cd735f434fce None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:22.418013 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee555d04-9443-4071-8d6c-cd735f434fce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:22.418311 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee555d04-9443-4071-8d6c-cd735f434fce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:22.419694 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee555d04-9443-4071-8d6c-cd735f434fce None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:22.428493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 552/1097] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:22.434261 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee555d04-9443-4071-8d6c-cd735f434fce None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:22.501277 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78874b3-1aea-4b7b-8adb-fc3e48f133b8 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:22.528329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:22.538283 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7b936-e9db-402b-997b-476f023af602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:22.539098 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7b936-e9db-402b-997b-476f023af602 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:22.539299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7b936-e9db-402b-997b-476f023af602 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:22.539566 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7b936-e9db-402b-997b-476f023af602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:22.540681 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee555d04-9443-4071-8d6c-cd735f434fce None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:22.549702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7b936-e9db-402b-997b-476f023af602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:22.551791 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7b936-e9db-402b-997b-476f023af602 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:22.565871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-50a7b936-e9db-402b-997b-476f023af602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:02:22.565871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:22.565871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:22.565871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:22.565871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:22.565871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:22.565871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:22.565871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:22.565871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:22.565871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:22.565871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:22.565871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:22.565871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:02:22.565871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:02:22.565871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:02:22.565871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:02:22.567240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:22.567240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:22.567240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:22.567240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:22.567240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:22.567240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:22.567240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:22.567240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:22.567240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:22.567240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:22.567240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:22.567240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:22.567240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:22.567240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:22.567240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:22.567240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:22.567240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:02:22.568225 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:02:22.568225 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:02:22.568225 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:22.568225 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 546/1098] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:02:22.572994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7b936-e9db-402b-997b-476f023af602 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:22.576967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69605d5-91d3-48a1-9581-b17ec870b247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:22.577596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69605d5-91d3-48a1-9581-b17ec870b247 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:22.577867 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69605d5-91d3-48a1-9581-b17ec870b247 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:22.578130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69605d5-91d3-48a1-9581-b17ec870b247 None admin] PATH_INFO: `/v3/users/5f141a2abdbf417fa5be5a15f5db40fd` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:22.579260 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69605d5-91d3-48a1-9581-b17ec870b247 None admin] RBAC: Authorizing `identity:get_user(user_id=5f141a2abdbf417fa5be5a15f5db40fd)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:22.580658 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69605d5-91d3-48a1-9581-b17ec870b247 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:22.582897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 553/1099] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:22 2022] GET /identity/v3/users/5f141a2abdbf417fa5be5a15f5db40fd => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:22.588589 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69605d5-91d3-48a1-9581-b17ec870b247 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:22.645345 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:22.674472 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7b936-e9db-402b-997b-476f023af602 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:22.693208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69605d5-91d3-48a1-9581-b17ec870b247 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:22.709941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8979c9-8f92-4620-9b7e-c2ae795477e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:22.710562 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8979c9-8f92-4620-9b7e-c2ae795477e0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:22.710871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8979c9-8f92-4620-9b7e-c2ae795477e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:22.711141 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8979c9-8f92-4620-9b7e-c2ae795477e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:22.711784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8979c9-8f92-4620-9b7e-c2ae795477e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:22.712087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8979c9-8f92-4620-9b7e-c2ae795477e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:22.713518 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8979c9-8f92-4620-9b7e-c2ae795477e0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:22.726129 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 547/1100] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:22.728001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d60cd4-1d6f-4269-ae63-c59831efa1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:22.728602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d60cd4-1d6f-4269-ae63-c59831efa1ea None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:22.728878 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d60cd4-1d6f-4269-ae63-c59831efa1ea None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:22.729147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d60cd4-1d6f-4269-ae63-c59831efa1ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:22.731790 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8979c9-8f92-4620-9b7e-c2ae795477e0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:22.777991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:22.800035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d60cd4-1d6f-4269-ae63-c59831efa1ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:22.801853 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d60cd4-1d6f-4269-ae63-c59831efa1ea None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:22.837366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8979c9-8f92-4620-9b7e-c2ae795477e0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:22.858514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:22.871914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7a862-69e1-4d4b-b8c4-0ef438e6c73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:22.872555 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7a862-69e1-4d4b-b8c4-0ef438e6c73f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:22.872834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7a862-69e1-4d4b-b8c4-0ef438e6c73f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:22.873105 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7a862-69e1-4d4b-b8c4-0ef438e6c73f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:22.880372 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7a862-69e1-4d4b-b8c4-0ef438e6c73f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:22.881864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7a862-69e1-4d4b-b8c4-0ef438e6c73f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:22.893334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-abd7a862-69e1-4d4b-b8c4-0ef438e6c73f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:02:22.893334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:22.893334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:22.893334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:22.893334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:22.893334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:22.893334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:22.893334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:22.893334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:22.893334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:22.893334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:22.893334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:22.893334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:02:22.893334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:02:22.893334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:02:22.894387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:02:22.894387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:22.894387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:22.894387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:22.894387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:22.894387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:22.894387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:22.894387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:22.894387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:22.894387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:22.894387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:22.894387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:22.894387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:22.894387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:22.894387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:22.894387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:22.894387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:22.896037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:02:22.896037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:02:22.896037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:02:22.896037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:02:22.896037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:02:22.896037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:02:22.896037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:02:22.896037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:22.896037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 548/1101] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:02:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:02:22.900948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7a862-69e1-4d4b-b8c4-0ef438e6c73f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:22.910261 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 554/1102] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:02:22.916336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d60cd4-1d6f-4269-ae63-c59831efa1ea None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:23.001954 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:23.005676 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7a862-69e1-4d4b-b8c4-0ef438e6c73f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:23.017376 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d60cd4-1d6f-4269-ae63-c59831efa1ea None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:23.043120 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bbe36-0047-4cf4-9a18-f8d71736be27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:23.043725 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bbe36-0047-4cf4-9a18-f8d71736be27 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:23.043998 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bbe36-0047-4cf4-9a18-f8d71736be27 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:23.044271 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bbe36-0047-4cf4-9a18-f8d71736be27 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:23.044947 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bbe36-0047-4cf4-9a18-f8d71736be27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:23.045247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bbe36-0047-4cf4-9a18-f8d71736be27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:23.046683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bbe36-0047-4cf4-9a18-f8d71736be27 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:23.052322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46850b1-c536-431c-b132-ec99a3c1610b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:23.052948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46850b1-c536-431c-b132-ec99a3c1610b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:23.053229 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46850b1-c536-431c-b132-ec99a3c1610b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:23.053503 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46850b1-c536-431c-b132-ec99a3c1610b None admin] PATH_INFO: `/v3/projects/b0eb7b2efa6c4d7382a8ad7ec9363d61` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:23.054618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46850b1-c536-431c-b132-ec99a3c1610b None admin] RBAC: Authorizing `identity:get_project(project_id=b0eb7b2efa6c4d7382a8ad7ec9363d61)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:23.056073 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46850b1-c536-431c-b132-ec99a3c1610b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:23.057716 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 555/1103] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:22 2022] GET /identity/v3/projects/b0eb7b2efa6c4d7382a8ad7ec9363d61 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:23.059023 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 549/1104] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:02:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:23.064618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46850b1-c536-431c-b132-ec99a3c1610b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:23.064783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bbe36-0047-4cf4-9a18-f8d71736be27 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:23.103144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:23.172180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bbe36-0047-4cf4-9a18-f8d71736be27 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:23.183779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46850b1-c536-431c-b132-ec99a3c1610b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:23.208718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4607764-664a-47de-b331-e9eab297e4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:23.209328 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4607764-664a-47de-b331-e9eab297e4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:23.209610 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4607764-664a-47de-b331-e9eab297e4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:23.209884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4607764-664a-47de-b331-e9eab297e4e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:23.210611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4607764-664a-47de-b331-e9eab297e4e2 None admin] RBAC: Adding query filter params (role.id=91f753ea35ce4f5a953d0a503481da1a, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=6afd4f6ffdae4b16aa5ba757e68c18c4) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:23.210949 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4607764-664a-47de-b331-e9eab297e4e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:23.212379 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4607764-664a-47de-b331-e9eab297e4e2 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:23.218418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 550/1105] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:23 2022] GET /identity/v3/role_assignments?user.id=6afd4f6ffdae4b16aa5ba757e68c18c4&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=91f753ea35ce4f5a953d0a503481da1a => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:23.218816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd573f47-94ba-4c2d-af56-7eecb107ade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:23.219529 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd573f47-94ba-4c2d-af56-7eecb107ade5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:23.219810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd573f47-94ba-4c2d-af56-7eecb107ade5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:23.220074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd573f47-94ba-4c2d-af56-7eecb107ade5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:23.220825 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd573f47-94ba-4c2d-af56-7eecb107ade5 None admin] RBAC: Adding query filter params (role.id=ba05677650e046a48d5f19440d48fbf2, scope.project.id=b0eb7b2efa6c4d7382a8ad7ec9363d61, user.id=5f141a2abdbf417fa5be5a15f5db40fd) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:23.221128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd573f47-94ba-4c2d-af56-7eecb107ade5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:23.222546 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd573f47-94ba-4c2d-af56-7eecb107ade5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:23.235851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 556/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:23 2022] GET /identity/v3/role_assignments?user.id=5f141a2abdbf417fa5be5a15f5db40fd&scope.project.id=b0eb7b2efa6c4d7382a8ad7ec9363d61&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:02:25.237015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e410ea3-d46f-4ab8-a278-1b7ae90b913d None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:25.237545 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e410ea3-d46f-4ab8-a278-1b7ae90b913d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:25.237545 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e410ea3-d46f-4ab8-a278-1b7ae90b913d None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:25.239532 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 551/1107] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:25.244822 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fdf90-6cdb-4cc7-9049-ba4cf8e9ba7d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:25.245239 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fdf90-6cdb-4cc7-9049-ba4cf8e9ba7d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:25.245363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fdf90-6cdb-4cc7-9049-ba4cf8e9ba7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:25.264178 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-632fdf90-6cdb-4cc7-9049-ba4cf8e9ba7d None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:25.265350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-632fdf90-6cdb-4cc7-9049-ba4cf8e9ba7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:25.334006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 557/1108] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:25.608239 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e410ea3-d46f-4ab8-a278-1b7ae90b913d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:25.633169 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:25.714809 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e410ea3-d46f-4ab8-a278-1b7ae90b913d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:25.750570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f10ee-4390-4e82-975e-d4de46fa83f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:25.751231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f10ee-4390-4e82-975e-d4de46fa83f8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:25.751507 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f10ee-4390-4e82-975e-d4de46fa83f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:25.751776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f10ee-4390-4e82-975e-d4de46fa83f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:25.752884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f10ee-4390-4e82-975e-d4de46fa83f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:25.754976 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f10ee-4390-4e82-975e-d4de46fa83f8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:25.756574 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 552/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:25.762452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fdf90-6cdb-4cc7-9049-ba4cf8e9ba7d None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:25.780609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:25.865030 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fdf90-6cdb-4cc7-9049-ba4cf8e9ba7d None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:25.900575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d7056-c2b6-43a7-b72b-ad61e1d825f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:25.901178 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d7056-c2b6-43a7-b72b-ad61e1d825f0 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:25.901455 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d7056-c2b6-43a7-b72b-ad61e1d825f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:25.901725 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d7056-c2b6-43a7-b72b-ad61e1d825f0 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:25.902370 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d7056-c2b6-43a7-b72b-ad61e1d825f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:25.903820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d7056-c2b6-43a7-b72b-ad61e1d825f0 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:25.925310 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 558/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 6 16:02:25 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:27.919220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc6365-07cd-4123-9276-153441aebe4a None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:27.919749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc6365-07cd-4123-9276-153441aebe4a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:27.919749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc6365-07cd-4123-9276-153441aebe4a None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:27.921012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 553/1111] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:27.927039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cc4b9-7331-407c-aada-60f8b77bf506 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:27.927474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cc4b9-7331-407c-aada-60f8b77bf506 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:27.927578 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cc4b9-7331-407c-aada-60f8b77bf506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:27.946222 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-5b8cc4b9-7331-407c-aada-60f8b77bf506 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:27.947458 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5b8cc4b9-7331-407c-aada-60f8b77bf506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:28.012842 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 559/1112] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:28.299223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc6365-07cd-4123-9276-153441aebe4a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:28.395657 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:28.405499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc6365-07cd-4123-9276-153441aebe4a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:28.439508 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6315c5-6f20-4e67-b597-c4df795a42d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:28.440134 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6315c5-6f20-4e67-b597-c4df795a42d9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:28.440411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6315c5-6f20-4e67-b597-c4df795a42d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:28.440683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6315c5-6f20-4e67-b597-c4df795a42d9 None admin] PATH_INFO: `/v3/roles/bad9ff35005240e99bedef3fddb1dc77` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:28.442551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6315c5-6f20-4e67-b597-c4df795a42d9 None admin] RBAC: Authorizing `identity:get_role(role_id=bad9ff35005240e99bedef3fddb1dc77)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:28.443946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6315c5-6f20-4e67-b597-c4df795a42d9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:28.445268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 554/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:28 2022] GET /identity/v3/roles/bad9ff35005240e99bedef3fddb1dc77 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:28.451154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cc4b9-7331-407c-aada-60f8b77bf506 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:28.453331 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:28.554689 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cc4b9-7331-407c-aada-60f8b77bf506 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:28.590840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420972d-63e2-496e-8dac-4ea1a147c959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:28.591462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420972d-63e2-496e-8dac-4ea1a147c959 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:28.591739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420972d-63e2-496e-8dac-4ea1a147c959 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:28.592009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420972d-63e2-496e-8dac-4ea1a147c959 None admin] PATH_INFO: `/v3/users/b602064eb9e24b9cb5396e076260393a` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:28.605349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420972d-63e2-496e-8dac-4ea1a147c959 None admin] RBAC: Authorizing `identity:get_user(user_id=b602064eb9e24b9cb5396e076260393a)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:28.606816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420972d-63e2-496e-8dac-4ea1a147c959 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:28.608471 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 560/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:28 2022] GET /identity/v3/users/b602064eb9e24b9cb5396e076260393a => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:28.614754 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6315c5-6f20-4e67-b597-c4df795a42d9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:28.715821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6315c5-6f20-4e67-b597-c4df795a42d9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:28.750291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc260834-e6db-4a53-8b87-054424dc23e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:28.750920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc260834-e6db-4a53-8b87-054424dc23e6 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:28.751196 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc260834-e6db-4a53-8b87-054424dc23e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:28.751466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc260834-e6db-4a53-8b87-054424dc23e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:28.779551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:28.821111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc260834-e6db-4a53-8b87-054424dc23e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:28.822477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc260834-e6db-4a53-8b87-054424dc23e6 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:28.921026 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 555/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:02:28.927856 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420972d-63e2-496e-8dac-4ea1a147c959 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:28.986198 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:29.028724 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420972d-63e2-496e-8dac-4ea1a147c959 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:29.062928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffec761-8bd7-4c2e-8d9b-4a1aec82cc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:29.063551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffec761-8bd7-4c2e-8d9b-4a1aec82cc8b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:29.063830 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffec761-8bd7-4c2e-8d9b-4a1aec82cc8b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:29.064100 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffec761-8bd7-4c2e-8d9b-4a1aec82cc8b None admin] PATH_INFO: `/v3/projects/b0eb7b2efa6c4d7382a8ad7ec9363d61` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:29.065196 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffec761-8bd7-4c2e-8d9b-4a1aec82cc8b None admin] RBAC: Authorizing `identity:get_project(project_id=b0eb7b2efa6c4d7382a8ad7ec9363d61)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:29.067291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffec761-8bd7-4c2e-8d9b-4a1aec82cc8b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:29.068932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 561/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:28 2022] GET /identity/v3/projects/b0eb7b2efa6c4d7382a8ad7ec9363d61 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:29.075267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc260834-e6db-4a53-8b87-054424dc23e6 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:29.132573 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:29.172829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc260834-e6db-4a53-8b87-054424dc23e6 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:29.207291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ad07c-8f21-46a6-b590-669f875a351c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:29.207902 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ad07c-8f21-46a6-b590-669f875a351c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:29.208172 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ad07c-8f21-46a6-b590-669f875a351c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:29.208437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ad07c-8f21-46a6-b590-669f875a351c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:29.209160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ad07c-8f21-46a6-b590-669f875a351c None admin] RBAC: Adding query filter params (role.id=bad9ff35005240e99bedef3fddb1dc77, scope.project.id=b0eb7b2efa6c4d7382a8ad7ec9363d61, user.id=b602064eb9e24b9cb5396e076260393a) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:29.209459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ad07c-8f21-46a6-b590-669f875a351c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:29.210911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ad07c-8f21-46a6-b590-669f875a351c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:29.216472 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 556/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:29 2022] GET /identity/v3/role_assignments?user.id=b602064eb9e24b9cb5396e076260393a&scope.project.id=b0eb7b2efa6c4d7382a8ad7ec9363d61&role.id=bad9ff35005240e99bedef3fddb1dc77 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:31.113646 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b707b-80ad-461e-b280-f82ae32cf741 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:31.114181 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b707b-80ad-461e-b280-f82ae32cf741 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:31.114349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b707b-80ad-461e-b280-f82ae32cf741 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:31.115739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 562/1118] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:31.120881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f5846-efe5-45d9-a1e4-ade52997f2d1 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:31.121267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f5846-efe5-45d9-a1e4-ade52997f2d1 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:31.121422 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f5846-efe5-45d9-a1e4-ade52997f2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:31.140337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-974f5846-efe5-45d9-a1e4-ade52997f2d1 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:31.141521 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-974f5846-efe5-45d9-a1e4-ade52997f2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:31.208372 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 557/1119] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:31.481638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b707b-80ad-461e-b280-f82ae32cf741 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:31.584661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b707b-80ad-461e-b280-f82ae32cf741 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:31.611569 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:31.620402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc29a25-6c06-4dbd-a583-260927d5c73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:31.621027 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc29a25-6c06-4dbd-a583-260927d5c73e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:31.621307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc29a25-6c06-4dbd-a583-260927d5c73e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:31.621578 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc29a25-6c06-4dbd-a583-260927d5c73e None admin] PATH_INFO: `/v3/roles/bad9ff35005240e99bedef3fddb1dc77` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:31.622992 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc29a25-6c06-4dbd-a583-260927d5c73e None admin] RBAC: Authorizing `identity:get_role(role_id=bad9ff35005240e99bedef3fddb1dc77)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:31.624341 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc29a25-6c06-4dbd-a583-260927d5c73e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:31.625668 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 563/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:31 2022] GET /identity/v3/roles/bad9ff35005240e99bedef3fddb1dc77 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:31.632282 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f5846-efe5-45d9-a1e4-ade52997f2d1 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:31.686184 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:31.730825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f5846-efe5-45d9-a1e4-ade52997f2d1 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:31.766360 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5254f8-831d-4027-9811-3d2ee9760f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:31.766937 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5254f8-831d-4027-9811-3d2ee9760f8f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:31.767219 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5254f8-831d-4027-9811-3d2ee9760f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:31.767491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5254f8-831d-4027-9811-3d2ee9760f8f None admin] PATH_INFO: `/v3/users/b602064eb9e24b9cb5396e076260393a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:31.768605 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5254f8-831d-4027-9811-3d2ee9760f8f None admin] RBAC: Authorizing `identity:get_user(user_id=b602064eb9e24b9cb5396e076260393a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:31.770249 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5254f8-831d-4027-9811-3d2ee9760f8f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:31.771869 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 558/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:31 2022] GET /identity/v3/users/b602064eb9e24b9cb5396e076260393a => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:31.777732 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc29a25-6c06-4dbd-a583-260927d5c73e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:31.887470 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc29a25-6c06-4dbd-a583-260927d5c73e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:31.921834 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b8cfc-4853-450a-8fb0-484ab530e99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:31.922462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b8cfc-4853-450a-8fb0-484ab530e99c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:31.922775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b8cfc-4853-450a-8fb0-484ab530e99c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:31.923051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b8cfc-4853-450a-8fb0-484ab530e99c None admin] PATH_INFO: `/v3/projects/b0eb7b2efa6c4d7382a8ad7ec9363d61` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:31.924150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b8cfc-4853-450a-8fb0-484ab530e99c None admin] RBAC: Authorizing `identity:get_project(project_id=b0eb7b2efa6c4d7382a8ad7ec9363d61)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:31.925564 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b8cfc-4853-450a-8fb0-484ab530e99c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:31.928210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 564/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:31 2022] GET /identity/v3/projects/b0eb7b2efa6c4d7382a8ad7ec9363d61 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:31.934205 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5254f8-831d-4027-9811-3d2ee9760f8f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:32.030909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5254f8-831d-4027-9811-3d2ee9760f8f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:32.054652 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:32.067348 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b3080-b8f2-43fb-bee1-5440be7f76c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:32.067951 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b3080-b8f2-43fb-bee1-5440be7f76c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:32.068233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b3080-b8f2-43fb-bee1-5440be7f76c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:32.068508 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b3080-b8f2-43fb-bee1-5440be7f76c4 None admin] PATH_INFO: `/v3/projects/b0eb7b2efa6c4d7382a8ad7ec9363d61/users/b602064eb9e24b9cb5396e076260393a/roles/bad9ff35005240e99bedef3fddb1dc77` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:32.070654 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b3080-b8f2-43fb-bee1-5440be7f76c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0eb7b2efa6c4d7382a8ad7ec9363d61, user_id=b602064eb9e24b9cb5396e076260393a, role_id=bad9ff35005240e99bedef3fddb1dc77)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:32.072192 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b3080-b8f2-43fb-bee1-5440be7f76c4 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:32.086335 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 559/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:02:31 2022] PUT /identity/v3/projects/b0eb7b2efa6c4d7382a8ad7ec9363d61/users/b602064eb9e24b9cb5396e076260393a/roles/bad9ff35005240e99bedef3fddb1dc77 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:02:33.992506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d704720-5fc7-4aa2-9761-e606a295586a None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:33.993008 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d704720-5fc7-4aa2-9761-e606a295586a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:33.993008 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d704720-5fc7-4aa2-9761-e606a295586a None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:33.994308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 565/1124] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:33.999703 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707f645-3a8b-4138-8c01-dbd7f7edd12a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:34.000123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707f645-3a8b-4138-8c01-dbd7f7edd12a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:34.000238 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707f645-3a8b-4138-8c01-dbd7f7edd12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:34.018562 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9707f645-3a8b-4138-8c01-dbd7f7edd12a None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:34.019894 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9707f645-3a8b-4138-8c01-dbd7f7edd12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:34.088865 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 560/1125] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:34.359350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d704720-5fc7-4aa2-9761-e606a295586a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:34.422806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:34.463672 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d704720-5fc7-4aa2-9761-e606a295586a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:34.497383 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393c646-b91a-4ae9-a210-a17ace116b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:34.497995 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393c646-b91a-4ae9-a210-a17ace116b4a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:34.498271 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393c646-b91a-4ae9-a210-a17ace116b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:34.498541 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393c646-b91a-4ae9-a210-a17ace116b4a None admin] PATH_INFO: `/v3/roles/bad9ff35005240e99bedef3fddb1dc77` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:34.499942 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b393c646-b91a-4ae9-a210-a17ace116b4a None admin] RBAC: Authorizing `identity:get_role(role_id=bad9ff35005240e99bedef3fddb1dc77)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:34.501292 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b393c646-b91a-4ae9-a210-a17ace116b4a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:34.502619 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 566/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:34 2022] GET /identity/v3/roles/bad9ff35005240e99bedef3fddb1dc77 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:34.509221 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707f645-3a8b-4138-8c01-dbd7f7edd12a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:34.606566 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707f645-3a8b-4138-8c01-dbd7f7edd12a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:34.620403 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:34.642425 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5077fd-7836-4728-bd06-d36d1a485a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:34.643074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5077fd-7836-4728-bd06-d36d1a485a42 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:34.643353 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5077fd-7836-4728-bd06-d36d1a485a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:34.643624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5077fd-7836-4728-bd06-d36d1a485a42 None admin] PATH_INFO: `/v3/users/b602064eb9e24b9cb5396e076260393a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:34.644820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5077fd-7836-4728-bd06-d36d1a485a42 None admin] RBAC: Authorizing `identity:get_user(user_id=b602064eb9e24b9cb5396e076260393a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:34.646249 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5077fd-7836-4728-bd06-d36d1a485a42 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:34.647865 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 561/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:34 2022] GET /identity/v3/users/b602064eb9e24b9cb5396e076260393a => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:34.653677 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393c646-b91a-4ae9-a210-a17ace116b4a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:34.751347 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393c646-b91a-4ae9-a210-a17ace116b4a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:34.780493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:34.786452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8751464-caa3-4cb5-90db-3c634f2a4e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:34.787083 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8751464-caa3-4cb5-90db-3c634f2a4e25 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:34.787366 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8751464-caa3-4cb5-90db-3c634f2a4e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:34.787635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8751464-caa3-4cb5-90db-3c634f2a4e25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:34.857338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8751464-caa3-4cb5-90db-3c634f2a4e25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:34.858807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8751464-caa3-4cb5-90db-3c634f2a4e25 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:34.958888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 567/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:02:34.964854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5077fd-7836-4728-bd06-d36d1a485a42 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:35.063171 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5077fd-7836-4728-bd06-d36d1a485a42 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:35.097566 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe56e19-582b-4cf3-8ef5-46302609b342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:35.098195 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe56e19-582b-4cf3-8ef5-46302609b342 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:35.098480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe56e19-582b-4cf3-8ef5-46302609b342 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:35.098790 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe56e19-582b-4cf3-8ef5-46302609b342 None admin] PATH_INFO: `/v3/projects/b0eb7b2efa6c4d7382a8ad7ec9363d61` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:35.099904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe56e19-582b-4cf3-8ef5-46302609b342 None admin] RBAC: Authorizing `identity:get_project(project_id=b0eb7b2efa6c4d7382a8ad7ec9363d61)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:35.101342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe56e19-582b-4cf3-8ef5-46302609b342 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:35.103016 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 562/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:34 2022] GET /identity/v3/projects/b0eb7b2efa6c4d7382a8ad7ec9363d61 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:35.109386 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8751464-caa3-4cb5-90db-3c634f2a4e25 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:35.140935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:35.212776 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8751464-caa3-4cb5-90db-3c634f2a4e25 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:35.247436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975224f-118c-43e1-a509-9d8b3001043b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:35.248037 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975224f-118c-43e1-a509-9d8b3001043b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:35.248315 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975224f-118c-43e1-a509-9d8b3001043b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:35.248585 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975224f-118c-43e1-a509-9d8b3001043b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:35.249303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975224f-118c-43e1-a509-9d8b3001043b None admin] RBAC: Adding query filter params (role.id=bad9ff35005240e99bedef3fddb1dc77, scope.project.id=b0eb7b2efa6c4d7382a8ad7ec9363d61, user.id=b602064eb9e24b9cb5396e076260393a) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:35.249610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975224f-118c-43e1-a509-9d8b3001043b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:35.251054 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975224f-118c-43e1-a509-9d8b3001043b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:35.257834 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 568/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:35 2022] GET /identity/v3/role_assignments?user.id=b602064eb9e24b9cb5396e076260393a&scope.project.id=b0eb7b2efa6c4d7382a8ad7ec9363d61&role.id=bad9ff35005240e99bedef3fddb1dc77 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:37.221171 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6bc5d-bd22-4543-9189-90a3ef7f2774 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:37.221724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6bc5d-bd22-4543-9189-90a3ef7f2774 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:37.221724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6bc5d-bd22-4543-9189-90a3ef7f2774 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:37.223743 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 563/1131] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:37.228999 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d104d0-4115-42d2-8f61-8aabb9e2dcc0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:37.229402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d104d0-4115-42d2-8f61-8aabb9e2dcc0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:37.229550 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d104d0-4115-42d2-8f61-8aabb9e2dcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:37.248860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b9d104d0-4115-42d2-8f61-8aabb9e2dcc0 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:37.250039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b9d104d0-4115-42d2-8f61-8aabb9e2dcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:37.317165 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 569/1132] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:37.587893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6bc5d-bd22-4543-9189-90a3ef7f2774 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:37.637947 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:37.690651 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6bc5d-bd22-4543-9189-90a3ef7f2774 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:37.728656 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185288e2-5887-44b3-b3a3-2b2b0342aff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:37.729304 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185288e2-5887-44b3-b3a3-2b2b0342aff0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:37.729580 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185288e2-5887-44b3-b3a3-2b2b0342aff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:37.729881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185288e2-5887-44b3-b3a3-2b2b0342aff0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:37.731027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185288e2-5887-44b3-b3a3-2b2b0342aff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:37.732469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185288e2-5887-44b3-b3a3-2b2b0342aff0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:37.734045 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 564/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:37.740128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d104d0-4115-42d2-8f61-8aabb9e2dcc0 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:37.766099 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:37.846563 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d104d0-4115-42d2-8f61-8aabb9e2dcc0 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:37.882353 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeaaf32-15ce-4b93-b989-d7f635efe9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:37.882985 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeaaf32-15ce-4b93-b989-d7f635efe9cd None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:37.883262 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeaaf32-15ce-4b93-b989-d7f635efe9cd None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:37.883530 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeaaf32-15ce-4b93-b989-d7f635efe9cd None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:37.884169 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeaaf32-15ce-4b93-b989-d7f635efe9cd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:37.885581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeaaf32-15ce-4b93-b989-d7f635efe9cd None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:37.914432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 570/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 6 16:02:37 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:39.883914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470838f-27df-40e4-a64d-3a55ee5c1d1f None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:39.884432 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470838f-27df-40e4-a64d-3a55ee5c1d1f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:39.884432 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470838f-27df-40e4-a64d-3a55ee5c1d1f None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:39.885725 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 565/1135] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:39.890862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b33e3e-2e0f-4fec-886b-7e4e8e875574 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:39.891235 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b33e3e-2e0f-4fec-886b-7e4e8e875574 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:39.891413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b33e3e-2e0f-4fec-886b-7e4e8e875574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:39.909681 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-93b33e3e-2e0f-4fec-886b-7e4e8e875574 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:39.911826 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-93b33e3e-2e0f-4fec-886b-7e4e8e875574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:39.978413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 571/1136] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:40.252630 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470838f-27df-40e4-a64d-3a55ee5c1d1f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:40.360955 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470838f-27df-40e4-a64d-3a55ee5c1d1f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:40.384918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:40.403241 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb482ad-7a64-4b4a-be00-722bec5656fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:40.403861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb482ad-7a64-4b4a-be00-722bec5656fb None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:40.404133 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb482ad-7a64-4b4a-be00-722bec5656fb None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:40.404400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb482ad-7a64-4b4a-be00-722bec5656fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:40.405504 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb482ad-7a64-4b4a-be00-722bec5656fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:40.406963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb482ad-7a64-4b4a-be00-722bec5656fb None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:40.408518 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 566/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:02:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:40.415475 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b33e3e-2e0f-4fec-886b-7e4e8e875574 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:40.425596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:40.517894 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b33e3e-2e0f-4fec-886b-7e4e8e875574 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:40.552901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade4a47-a20f-4439-a14f-988e699d165f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:40.553501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade4a47-a20f-4439-a14f-988e699d165f None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:40.553780 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade4a47-a20f-4439-a14f-988e699d165f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:40.554049 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade4a47-a20f-4439-a14f-988e699d165f None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:40.554725 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade4a47-a20f-4439-a14f-988e699d165f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:40.556146 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade4a47-a20f-4439-a14f-988e699d165f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:40.579637 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 572/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 6 16:02:40 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:42.550763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca30d27-edcb-4807-8c06-7329f750435b None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:42.552093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca30d27-edcb-4807-8c06-7329f750435b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:42.552093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca30d27-edcb-4807-8c06-7329f750435b None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:42.553387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 567/1139] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:42.558587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5711b86-3e83-4b7b-a22f-312de7c20e28 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:42.558977 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5711b86-3e83-4b7b-a22f-312de7c20e28 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:42.559186 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5711b86-3e83-4b7b-a22f-312de7c20e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:42.577201 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b5711b86-3e83-4b7b-a22f-312de7c20e28 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:42.578393 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b5711b86-3e83-4b7b-a22f-312de7c20e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:42.638324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:42.647753 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 573/1140] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:42.919560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca30d27-edcb-4807-8c06-7329f750435b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:43.019084 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca30d27-edcb-4807-8c06-7329f750435b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:43.054454 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc42e1-9696-45ad-a5b1-4925b608960a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:43.055141 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc42e1-9696-45ad-a5b1-4925b608960a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:43.055420 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc42e1-9696-45ad-a5b1-4925b608960a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:43.055692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc42e1-9696-45ad-a5b1-4925b608960a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:43.066784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc42e1-9696-45ad-a5b1-4925b608960a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:43.071645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-d4cc42e1-9696-45ad-a5b1-4925b608960a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:43.071645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:43.071645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:43.071645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:43.071645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:43.071645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:43.071645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:43.071645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:43.071645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:43.071645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:43.071645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:43.071645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:43.071645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:43.071645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:43.071645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:43.071645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:43.072751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:43.072751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:43.072751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:43.072751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:43.072751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:43.072751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:43.072751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:43.072751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:43.072751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:43.072751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:43.072751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:43.072751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:43.072751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:43.072751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:43.072751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:43.072751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:43.073599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:43.073599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:43.073599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:43.073599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:43.073599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:43.073599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:43.073599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:43.073599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:43.073599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:43.073599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:43.073599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:43.073599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:43.073599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 568/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:02:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:43.078884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5711b86-3e83-4b7b-a22f-312de7c20e28 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:43.210058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5711b86-3e83-4b7b-a22f-312de7c20e28 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:43.243114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e33e6-2d61-4746-8044-9f4b162f5aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:43.243704 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e33e6-2d61-4746-8044-9f4b162f5aca None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:43.243977 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e33e6-2d61-4746-8044-9f4b162f5aca None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:43.244246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e33e6-2d61-4746-8044-9f4b162f5aca None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:43.244896 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e33e6-2d61-4746-8044-9f4b162f5aca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:43.245193 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e33e6-2d61-4746-8044-9f4b162f5aca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:43.246538 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e33e6-2d61-4746-8044-9f4b162f5aca None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:43.254944 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 574/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:43.260790 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc42e1-9696-45ad-a5b1-4925b608960a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:43.307593 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:43.360315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc42e1-9696-45ad-a5b1-4925b608960a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:43.393596 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa07bd-ce3e-4b2f-935c-8c9b99c045c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:43.394223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa07bd-ce3e-4b2f-935c-8c9b99c045c5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:43.395085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa07bd-ce3e-4b2f-935c-8c9b99c045c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:43.395363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa07bd-ce3e-4b2f-935c-8c9b99c045c5 None admin] PATH_INFO: `/v3/users/ef1bff6ca0384e019fa6b23d79d5ae1c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:43.408224 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa07bd-ce3e-4b2f-935c-8c9b99c045c5 None admin] RBAC: Authorizing `identity:get_user(user_id=ef1bff6ca0384e019fa6b23d79d5ae1c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:43.409668 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa07bd-ce3e-4b2f-935c-8c9b99c045c5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:43.411320 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 569/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:43 2022] GET /identity/v3/users/ef1bff6ca0384e019fa6b23d79d5ae1c => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:43.417317 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e33e6-2d61-4746-8044-9f4b162f5aca None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:43.482758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:43.536878 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e33e6-2d61-4746-8044-9f4b162f5aca None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:43.578965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d839d5-390e-4d4e-989e-a07ed0fa9546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:43.579606 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d839d5-390e-4d4e-989e-a07ed0fa9546 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:43.579832 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d839d5-390e-4d4e-989e-a07ed0fa9546 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:43.580104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d839d5-390e-4d4e-989e-a07ed0fa9546 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:43.647451 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d839d5-390e-4d4e-989e-a07ed0fa9546 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:43.648877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d839d5-390e-4d4e-989e-a07ed0fa9546 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:43.702221 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:43.745584 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 575/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:02:43.751529 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa07bd-ce3e-4b2f-935c-8c9b99c045c5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:43.853821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa07bd-ce3e-4b2f-935c-8c9b99c045c5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:43.855434 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:43.888985 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b79d55-992b-4af4-a1ef-a9573b2d13f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:43.889617 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b79d55-992b-4af4-a1ef-a9573b2d13f0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:43.889897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b79d55-992b-4af4-a1ef-a9573b2d13f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:43.890173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b79d55-992b-4af4-a1ef-a9573b2d13f0 None admin] PATH_INFO: `/v3/projects/f6676b35e0ac42278d41f3df77160bd4` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:43.891354 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b79d55-992b-4af4-a1ef-a9573b2d13f0 None admin] RBAC: Authorizing `identity:get_project(project_id=f6676b35e0ac42278d41f3df77160bd4)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:43.892797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b79d55-992b-4af4-a1ef-a9573b2d13f0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:43.894510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 570/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:43 2022] GET /identity/v3/projects/f6676b35e0ac42278d41f3df77160bd4 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:43.900979 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d839d5-390e-4d4e-989e-a07ed0fa9546 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:44.001697 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d839d5-390e-4d4e-989e-a07ed0fa9546 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:44.034320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82618c2c-28be-4deb-9204-c558f7d958d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:44.034927 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82618c2c-28be-4deb-9204-c558f7d958d0 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:44.035822 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82618c2c-28be-4deb-9204-c558f7d958d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:44.036089 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82618c2c-28be-4deb-9204-c558f7d958d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:44.036799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82618c2c-28be-4deb-9204-c558f7d958d0 None admin] RBAC: Adding query filter params (role.id=ba05677650e046a48d5f19440d48fbf2, scope.project.id=f6676b35e0ac42278d41f3df77160bd4, user.id=ef1bff6ca0384e019fa6b23d79d5ae1c) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:44.037104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82618c2c-28be-4deb-9204-c558f7d958d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:44.038491 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82618c2c-28be-4deb-9204-c558f7d958d0 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:44.043704 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 576/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:43 2022] GET /identity/v3/role_assignments?user.id=ef1bff6ca0384e019fa6b23d79d5ae1c&scope.project.id=f6676b35e0ac42278d41f3df77160bd4&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:45.937089 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afca3d-b50e-4dbd-9770-71af1cbf685f None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:45.937589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afca3d-b50e-4dbd-9770-71af1cbf685f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:45.937589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afca3d-b50e-4dbd-9770-71af1cbf685f None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:45.938905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 571/1147] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:45.945037 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccded572-4ad0-4566-bf7e-57e1756a6e93 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:45.945454 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccded572-4ad0-4566-bf7e-57e1756a6e93 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:45.945591 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccded572-4ad0-4566-bf7e-57e1756a6e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:45.964191 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ccded572-4ad0-4566-bf7e-57e1756a6e93 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:45.965388 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ccded572-4ad0-4566-bf7e-57e1756a6e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:45.979852 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:46.032820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 577/1148] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:46.306991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afca3d-b50e-4dbd-9770-71af1cbf685f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:46.408250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afca3d-b50e-4dbd-9770-71af1cbf685f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:46.442807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a648fa-33a8-4561-9f16-9c0188957fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:46.443444 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a648fa-33a8-4561-9f16-9c0188957fdc None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:46.443718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a648fa-33a8-4561-9f16-9c0188957fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:46.443984 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a648fa-33a8-4561-9f16-9c0188957fdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:46.453914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a648fa-33a8-4561-9f16-9c0188957fdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:46.459287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-b9a648fa-33a8-4561-9f16-9c0188957fdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:46.459287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:46.459287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:46.459287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:46.459287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:46.459287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:46.459287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:46.459287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:46.459287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:46.459287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:46.459287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:46.459287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:46.459287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:46.459287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:46.459287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:46.459287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:46.460355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:46.460355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:46.460355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:46.460355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:46.460355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:46.460355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:46.460355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:46.460355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:46.460355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:46.460355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:46.460355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:46.460355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:46.460355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:46.460355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:46.460355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:46.460355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:46.461369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:46.461369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:46.461369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:46.461369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:46.461369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:46.461369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:46.461369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:46.461369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:46.461369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:46.461369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:46.461369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:46.461369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:46.461369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 572/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:02:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:46.466457 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccded572-4ad0-4566-bf7e-57e1756a6e93 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:46.568710 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccded572-4ad0-4566-bf7e-57e1756a6e93 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:46.604313 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409d7d9-5d14-42b6-a09d-223a80357a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:46.604925 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409d7d9-5d14-42b6-a09d-223a80357a06 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:46.605203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409d7d9-5d14-42b6-a09d-223a80357a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:46.605474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409d7d9-5d14-42b6-a09d-223a80357a06 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:46.606133 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409d7d9-5d14-42b6-a09d-223a80357a06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:46.606436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409d7d9-5d14-42b6-a09d-223a80357a06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:46.607830 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409d7d9-5d14-42b6-a09d-223a80357a06 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:46.616558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 578/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:46.622367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a648fa-33a8-4561-9f16-9c0188957fdc None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:46.653571 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:46.727187 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a648fa-33a8-4561-9f16-9c0188957fdc None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:46.760484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e05df3-d104-43d5-afa0-cc755e052f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:46.761111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e05df3-d104-43d5-afa0-cc755e052f87 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:46.761391 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e05df3-d104-43d5-afa0-cc755e052f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:46.761663 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e05df3-d104-43d5-afa0-cc755e052f87 None admin] PATH_INFO: `/v3/users/ef1bff6ca0384e019fa6b23d79d5ae1c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:46.762811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e05df3-d104-43d5-afa0-cc755e052f87 None admin] RBAC: Authorizing `identity:get_user(user_id=ef1bff6ca0384e019fa6b23d79d5ae1c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:46.764251 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e05df3-d104-43d5-afa0-cc755e052f87 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:46.765847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 573/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:46 2022] GET /identity/v3/users/ef1bff6ca0384e019fa6b23d79d5ae1c => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:46.772892 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409d7d9-5d14-42b6-a09d-223a80357a06 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:46.789233 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:46.874954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409d7d9-5d14-42b6-a09d-223a80357a06 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:46.908824 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ca549-3b42-4c7d-ba6d-912f7351b4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:46.909449 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ca549-3b42-4c7d-ba6d-912f7351b4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:46.909726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ca549-3b42-4c7d-ba6d-912f7351b4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:46.910026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ca549-3b42-4c7d-ba6d-912f7351b4f8 None admin] PATH_INFO: `/v3/projects/f6676b35e0ac42278d41f3df77160bd4` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:46.911212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ca549-3b42-4c7d-ba6d-912f7351b4f8 None admin] RBAC: Authorizing `identity:get_project(project_id=f6676b35e0ac42278d41f3df77160bd4)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:46.912635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ca549-3b42-4c7d-ba6d-912f7351b4f8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:46.914278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 579/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:46 2022] GET /identity/v3/projects/f6676b35e0ac42278d41f3df77160bd4 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:46.920263 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e05df3-d104-43d5-afa0-cc755e052f87 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:47.020955 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:47.025335 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e05df3-d104-43d5-afa0-cc755e052f87 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:47.060203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b914c5-bf00-4813-b47f-503925bf7df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:47.060853 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b914c5-bf00-4813-b47f-503925bf7df0 None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:47.061133 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b914c5-bf00-4813-b47f-503925bf7df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:47.061406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b914c5-bf00-4813-b47f-503925bf7df0 None admin] PATH_INFO: `/v3/projects/f6676b35e0ac42278d41f3df77160bd4/users/ef1bff6ca0384e019fa6b23d79d5ae1c/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:47.063300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b914c5-bf00-4813-b47f-503925bf7df0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6676b35e0ac42278d41f3df77160bd4, user_id=ef1bff6ca0384e019fa6b23d79d5ae1c, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:47.064740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b914c5-bf00-4813-b47f-503925bf7df0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:47.078380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 574/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:02:46 2022] PUT /identity/v3/projects/f6676b35e0ac42278d41f3df77160bd4/users/ef1bff6ca0384e019fa6b23d79d5ae1c/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:02:48.975020 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd463b-a743-4738-8814-82acdd4436c0 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:48.975548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd463b-a743-4738-8814-82acdd4436c0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:48.975548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd463b-a743-4738-8814-82acdd4436c0 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:48.976815 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 580/1154] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:48.981990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa31d2b-f6ab-4080-bbc9-5dbe99760c1d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:48.982379 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa31d2b-f6ab-4080-bbc9-5dbe99760c1d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:48.982530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa31d2b-f6ab-4080-bbc9-5dbe99760c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:49.000806 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-6fa31d2b-f6ab-4080-bbc9-5dbe99760c1d None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:49.001969 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6fa31d2b-f6ab-4080-bbc9-5dbe99760c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:49.067530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 575/1155] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:49.337317 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd463b-a743-4738-8814-82acdd4436c0 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:49.419778 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:49.440779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd463b-a743-4738-8814-82acdd4436c0 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:49.474879 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21126a3-caf0-4744-9082-87561f9f0861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:49.475505 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21126a3-caf0-4744-9082-87561f9f0861 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:49.475784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21126a3-caf0-4744-9082-87561f9f0861 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:49.476056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21126a3-caf0-4744-9082-87561f9f0861 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:49.487045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21126a3-caf0-4744-9082-87561f9f0861 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:02:49.492344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-a21126a3-caf0-4744-9082-87561f9f0861 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:49.492344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:49.492344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:49.492344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:49.492344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:49.492344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:49.492344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:49.492344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:49.492344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:49.492344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:49.492344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:49.492344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:49.492344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:02:49.492344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:02:49.492344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:02:49.492344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:02:49.493588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:02:49.493588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:02:49.493588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:02:49.493588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:02:49.493588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:49.493588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:49.493588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:49.493588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:49.493588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:49.493588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:49.493588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:49.493588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:49.493588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:49.493588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:49.493588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:49.493588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:49.494793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:49.494793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:49.494793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:49.494793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:49.494793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:02:49.494793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:02:49.494793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:02:49.494793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:02:49.494793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:02:49.494793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:02:49.494793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:02:49.494793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:02:49.494793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 581/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:02:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:49.499687 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa31d2b-f6ab-4080-bbc9-5dbe99760c1d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:49.586728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:49.599871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa31d2b-f6ab-4080-bbc9-5dbe99760c1d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:49.632833 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e643ba-25e0-4704-90ca-d0706be83e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:49.633435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e643ba-25e0-4704-90ca-d0706be83e83 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:49.633712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e643ba-25e0-4704-90ca-d0706be83e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:49.633979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e643ba-25e0-4704-90ca-d0706be83e83 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:49.634626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e643ba-25e0-4704-90ca-d0706be83e83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:49.634956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e643ba-25e0-4704-90ca-d0706be83e83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:49.636315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e643ba-25e0-4704-90ca-d0706be83e83 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:49.644164 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 576/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:02:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:49.650494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21126a3-caf0-4744-9082-87561f9f0861 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:49.750692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21126a3-caf0-4744-9082-87561f9f0861 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:49.784861 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e309f7-69e5-4ad5-8575-c9e621b3f8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:49.786595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e309f7-69e5-4ad5-8575-c9e621b3f8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:49.786905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e309f7-69e5-4ad5-8575-c9e621b3f8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:49.787178 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e309f7-69e5-4ad5-8575-c9e621b3f8e5 None admin] PATH_INFO: `/v3/users/ef1bff6ca0384e019fa6b23d79d5ae1c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:49.788299 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e309f7-69e5-4ad5-8575-c9e621b3f8e5 None admin] RBAC: Authorizing `identity:get_user(user_id=ef1bff6ca0384e019fa6b23d79d5ae1c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:49.789705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e309f7-69e5-4ad5-8575-c9e621b3f8e5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:49.791327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 582/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:02:49 2022] GET /identity/v3/users/ef1bff6ca0384e019fa6b23d79d5ae1c => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:49.797340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e643ba-25e0-4704-90ca-d0706be83e83 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:49.897269 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e643ba-25e0-4704-90ca-d0706be83e83 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:49.932226 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bfc0f-96fb-4bb7-af64-ea15401035b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:49.932834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bfc0f-96fb-4bb7-af64-ea15401035b7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:49.933111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bfc0f-96fb-4bb7-af64-ea15401035b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:49.933383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bfc0f-96fb-4bb7-af64-ea15401035b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:49.951834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:50.000410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bfc0f-96fb-4bb7-af64-ea15401035b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:50.001868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bfc0f-96fb-4bb7-af64-ea15401035b7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:50.099429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 577/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:02:50.105449 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e309f7-69e5-4ad5-8575-c9e621b3f8e5 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:50.117726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:50.209188 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e309f7-69e5-4ad5-8575-c9e621b3f8e5 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:50.243418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbe768-7ecb-45c1-86a5-8e0feb913121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:50.244041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbe768-7ecb-45c1-86a5-8e0feb913121 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:50.244320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbe768-7ecb-45c1-86a5-8e0feb913121 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:50.244589 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbe768-7ecb-45c1-86a5-8e0feb913121 None admin] PATH_INFO: `/v3/projects/f6676b35e0ac42278d41f3df77160bd4` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:50.245680 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbe768-7ecb-45c1-86a5-8e0feb913121 None admin] RBAC: Authorizing `identity:get_project(project_id=f6676b35e0ac42278d41f3df77160bd4)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:50.247132 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbe768-7ecb-45c1-86a5-8e0feb913121 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:50.248751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 583/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:02:50 2022] GET /identity/v3/projects/f6676b35e0ac42278d41f3df77160bd4 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:50.255216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bfc0f-96fb-4bb7-af64-ea15401035b7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:50.304442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:50.361368 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bfc0f-96fb-4bb7-af64-ea15401035b7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:50.398611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3b45a-5d3d-4808-a0c3-a298f13abc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:50.399231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3b45a-5d3d-4808-a0c3-a298f13abc92 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:50.399503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3b45a-5d3d-4808-a0c3-a298f13abc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:50.399768 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3b45a-5d3d-4808-a0c3-a298f13abc92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:50.400479 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3b45a-5d3d-4808-a0c3-a298f13abc92 None admin] RBAC: Adding query filter params (role.id=ba05677650e046a48d5f19440d48fbf2, scope.project.id=f6676b35e0ac42278d41f3df77160bd4, user.id=ef1bff6ca0384e019fa6b23d79d5ae1c) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:50.400774 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3b45a-5d3d-4808-a0c3-a298f13abc92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:50.402189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3b45a-5d3d-4808-a0c3-a298f13abc92 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:50.409744 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 578/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:02:50 2022] GET /identity/v3/role_assignments?user.id=ef1bff6ca0384e019fa6b23d79d5ae1c&scope.project.id=f6676b35e0ac42278d41f3df77160bd4&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:52.344964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9f025-00e4-4c2f-a8c8-7d6c15f31344 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:52.345504 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9f025-00e4-4c2f-a8c8-7d6c15f31344 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:52.345504 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9f025-00e4-4c2f-a8c8-7d6c15f31344 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:52.346902 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 584/1162] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:52.352856 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684d232-a0a3-42ec-a68a-6710b1282aa6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:52.353353 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684d232-a0a3-42ec-a68a-6710b1282aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:52.353451 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684d232-a0a3-42ec-a68a-6710b1282aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:52.373685 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-5684d232-a0a3-42ec-a68a-6710b1282aa6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:52.374933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5684d232-a0a3-42ec-a68a-6710b1282aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:52.440376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 579/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:52.712461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9f025-00e4-4c2f-a8c8-7d6c15f31344 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:52.792027 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:52.818740 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9f025-00e4-4c2f-a8c8-7d6c15f31344 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:52.853787 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf846a02-e1f1-470c-86c0-c4aaa301fd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:52.854321 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf846a02-e1f1-470c-86c0-c4aaa301fd7d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:52.854597 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf846a02-e1f1-470c-86c0-c4aaa301fd7d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:52.854897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf846a02-e1f1-470c-86c0-c4aaa301fd7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:52.925009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf846a02-e1f1-470c-86c0-c4aaa301fd7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:52.928499 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf846a02-e1f1-470c-86c0-c4aaa301fd7d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:53.002677 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:53.027852 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 585/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:02:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:02:53.033872 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684d232-a0a3-42ec-a68a-6710b1282aa6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:53.079440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:53.134122 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684d232-a0a3-42ec-a68a-6710b1282aa6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:53.167511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4cd18-18b3-47b9-b29c-858fd0c5121c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:53.168145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4cd18-18b3-47b9-b29c-858fd0c5121c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:53.168425 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4cd18-18b3-47b9-b29c-858fd0c5121c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:53.168696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4cd18-18b3-47b9-b29c-858fd0c5121c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:53.178906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4cd18-18b3-47b9-b29c-858fd0c5121c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:53.180368 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4cd18-18b3-47b9-b29c-858fd0c5121c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:53.193163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-14c4cd18-18b3-47b9-b29c-858fd0c5121c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 06 16:02:53.193163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:53.193163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 06 16:02:53.193163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 06 16:02:53.193163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:02:53.193163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:02:53.193163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:02:53.193163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:02:53.193163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 06 16:02:53.193163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:53.193163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:02:53.193163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:53.193163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:02:53.193163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:02:53.193163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:02:53.193163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:02:53.193163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:02:53.194400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:02:53.194400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:02:53.194400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:02:53.194400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:02:53.194400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:02:53.194400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:02:53.194400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:02:53.194400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:02:53.194400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:02:53.194400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:02:53.194400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:02:53.194400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:02:53.194400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:02:53.194400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:02:53.194400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:02:53.194400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:02:53.195329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:02:53.195329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:02:53.195329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:02:53.195329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:02:53.195329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:02:53.195329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:02:53.195329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:02:53.195329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:02:53.195329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:02:53.195329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:02:53.195329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 06 16:02:53.195329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:02:53.195329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 06 16:02:53.195329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:02:53.195329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 580/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 6 16:02:53 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:02:53.200450 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf846a02-e1f1-470c-86c0-c4aaa301fd7d None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:53.310079 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf846a02-e1f1-470c-86c0-c4aaa301fd7d None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:53.344497 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f1ef1-9e55-497c-b612-52f0c0669fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:53.345091 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f1ef1-9e55-497c-b612-52f0c0669fdb None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:53.345636 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f1ef1-9e55-497c-b612-52f0c0669fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:53.345636 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f1ef1-9e55-497c-b612-52f0c0669fdb None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:53.346239 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f1ef1-9e55-497c-b612-52f0c0669fdb None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:02:53.346537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f1ef1-9e55-497c-b612-52f0c0669fdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:53.348520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f1ef1-9e55-497c-b612-52f0c0669fdb None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:53.355530 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 586/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Sep 6 16:02:53 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:55.242183 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c8906-5643-4452-8244-21a4d1ed4bd0 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:55.242760 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c8906-5643-4452-8244-21a4d1ed4bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:55.242760 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c8906-5643-4452-8244-21a4d1ed4bd0 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:55.244072 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 581/1167] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:55.249442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781df30-5aa4-4a70-a9a8-f161625baeba None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:55.249848 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781df30-5aa4-4a70-a9a8-f161625baeba None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:55.249980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781df30-5aa4-4a70-a9a8-f161625baeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:55.268398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d781df30-5aa4-4a70-a9a8-f161625baeba None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:55.270177 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d781df30-5aa4-4a70-a9a8-f161625baeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:55.305832 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:55.340797 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 587/1168] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:02:55.610011 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c8906-5643-4452-8244-21a4d1ed4bd0 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:55.713044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c8906-5643-4452-8244-21a4d1ed4bd0 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:55.715421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:55.748363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d63b4-ab5f-41e6-9e19-452c48d38711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:55.748969 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d63b4-ab5f-41e6-9e19-452c48d38711 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:55.749251 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d63b4-ab5f-41e6-9e19-452c48d38711 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:55.749533 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d63b4-ab5f-41e6-9e19-452c48d38711 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:55.750197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d63b4-ab5f-41e6-9e19-452c48d38711 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:55.751689 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d63b4-ab5f-41e6-9e19-452c48d38711 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:55.768334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 582/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Tue Sep 6 16:02:55 2022] POST /identity/v3/domains => generated 267 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:02:57.772898 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3488ef5-c8c0-4ffd-8a8a-d819908f6c85 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:57.773468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3488ef5-c8c0-4ffd-8a8a-d819908f6c85 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:57.773468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3488ef5-c8c0-4ffd-8a8a-d819908f6c85 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:57.774812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 588/1170] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:02:57.780093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd56ac0-7fed-4197-8483-cf86ea309cbb None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:57.780492 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd56ac0-7fed-4197-8483-cf86ea309cbb None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:57.780660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd56ac0-7fed-4197-8483-cf86ea309cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:57.799276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8fd56ac0-7fed-4197-8483-cf86ea309cbb None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:02:57.800558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8fd56ac0-7fed-4197-8483-cf86ea309cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:02:57.867382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 583/1171] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:02:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:02:58.144939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3488ef5-c8c0-4ffd-8a8a-d819908f6c85 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:58.261649 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3488ef5-c8c0-4ffd-8a8a-d819908f6c85 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:58.296590 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0aa62f-4aff-48b4-a2f3-e5d8bafc0cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:58.297215 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0aa62f-4aff-48b4-a2f3-e5d8bafc0cdd None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:58.297496 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0aa62f-4aff-48b4-a2f3-e5d8bafc0cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:58.297772 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0aa62f-4aff-48b4-a2f3-e5d8bafc0cdd None admin] PATH_INFO: `/v3/domains/8f7935579e97495bb054e419ef8e15e6` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:58.308668 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0aa62f-4aff-48b4-a2f3-e5d8bafc0cdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=8f7935579e97495bb054e419ef8e15e6)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:58.310103 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0aa62f-4aff-48b4-a2f3-e5d8bafc0cdd None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:58.311756 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 589/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Tue Sep 6 16:02:58 2022] GET /identity/v3/domains/8f7935579e97495bb054e419ef8e15e6 => generated 267 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:02:58.318635 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd56ac0-7fed-4197-8483-cf86ea309cbb None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:02:58.412987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:02:58.419923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd56ac0-7fed-4197-8483-cf86ea309cbb None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:02:58.454032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37a7e5-9aeb-4114-a578-61e6c3ef89b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:02:58.454625 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37a7e5-9aeb-4114-a578-61e6c3ef89b2 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:02:58.454931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37a7e5-9aeb-4114-a578-61e6c3ef89b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:02:58.455196 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37a7e5-9aeb-4114-a578-61e6c3ef89b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:02:58.455830 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37a7e5-9aeb-4114-a578-61e6c3ef89b2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:02:58.457797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37a7e5-9aeb-4114-a578-61e6c3ef89b2 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:02:58.484072 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 584/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 6 16:02:58 2022] POST /identity/v3/projects => generated 361 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:03:00.482829 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157125c5-4d8a-41d0-8994-f941a39c7a14 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:00.483358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157125c5-4d8a-41d0-8994-f941a39c7a14 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:00.483358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157125c5-4d8a-41d0-8994-f941a39c7a14 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:00.484725 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 590/1174] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:00.490847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa540f94-0280-4e4e-a54b-6b6fdbcde2ce None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:00.491300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa540f94-0280-4e4e-a54b-6b6fdbcde2ce None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:00.491402 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa540f94-0280-4e4e-a54b-6b6fdbcde2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:00.509739 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-aa540f94-0280-4e4e-a54b-6b6fdbcde2ce None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:00.510989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-aa540f94-0280-4e4e-a54b-6b6fdbcde2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:00.580944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 585/1175] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:00.849229 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157125c5-4d8a-41d0-8994-f941a39c7a14 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:00.880917 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:00.951259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157125c5-4d8a-41d0-8994-f941a39c7a14 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:00.984241 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a55ebc-0ddf-401b-b4e6-049706536305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:00.984860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a55ebc-0ddf-401b-b4e6-049706536305 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:00.985134 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a55ebc-0ddf-401b-b4e6-049706536305 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:00.985403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a55ebc-0ddf-401b-b4e6-049706536305 None admin] PATH_INFO: `/v3/domains/8f7935579e97495bb054e419ef8e15e6` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:00.986511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a55ebc-0ddf-401b-b4e6-049706536305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8f7935579e97495bb054e419ef8e15e6)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:00.988456 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a55ebc-0ddf-401b-b4e6-049706536305 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:00.990048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 591/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Tue Sep 6 16:03:00 2022] GET /identity/v3/domains/8f7935579e97495bb054e419ef8e15e6 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:00.998614 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa540f94-0280-4e4e-a54b-6b6fdbcde2ce None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:01.079005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:01.104251 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa540f94-0280-4e4e-a54b-6b6fdbcde2ce None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:01.138315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74986bfb-c563-4709-9ed1-9bd98fcaae16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:01.138940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74986bfb-c563-4709-9ed1-9bd98fcaae16 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:01.139220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74986bfb-c563-4709-9ed1-9bd98fcaae16 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:01.139493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74986bfb-c563-4709-9ed1-9bd98fcaae16 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:01.140182 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74986bfb-c563-4709-9ed1-9bd98fcaae16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:01.141603 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74986bfb-c563-4709-9ed1-9bd98fcaae16 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:01.164626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 586/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 6 16:03:00 2022] POST /identity/v3/users => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:03:03.160785 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f282b94-fca0-429f-8719-34edb76a9e84 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:03.161652 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f282b94-fca0-429f-8719-34edb76a9e84 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:03.161933 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f282b94-fca0-429f-8719-34edb76a9e84 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:03.163329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 592/1178] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:03:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:03.168455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc178cc-53dd-44a1-8542-cad1ec131247 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:03.168852 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc178cc-53dd-44a1-8542-cad1ec131247 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:03.169010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc178cc-53dd-44a1-8542-cad1ec131247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:03.187840 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-2fc178cc-53dd-44a1-8542-cad1ec131247 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:03.189035 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2fc178cc-53dd-44a1-8542-cad1ec131247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:03.261192 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 587/1179] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:03.537330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f282b94-fca0-429f-8719-34edb76a9e84 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:03.635205 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:03.640454 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f282b94-fca0-429f-8719-34edb76a9e84 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:03.674676 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a83028-0e7f-4f8a-b32d-ee6c6ef09b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:03.675312 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a83028-0e7f-4f8a-b32d-ee6c6ef09b99 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:03.676575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a83028-0e7f-4f8a-b32d-ee6c6ef09b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:03.676854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a83028-0e7f-4f8a-b32d-ee6c6ef09b99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:03.686996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a83028-0e7f-4f8a-b32d-ee6c6ef09b99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:03.692340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-d7a83028-0e7f-4f8a-b32d-ee6c6ef09b99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:03.692340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:03.692340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:03.692340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:03.692340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:03.692340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:03.692340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:03.692340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:03.692340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:03.692340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:03.692340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:03.692340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:03.692340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:03.692340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:03.692340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:03.692340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:03.693465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:03.693465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:03.693465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:03.693465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:03.693465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:03.693465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:03.693465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:03.693465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:03.693465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:03.693465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:03.693465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:03.693465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:03.693465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:03.693465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:03.693465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:03.693465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:03.694705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:03.694705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:03.694705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:03.694705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:03.694705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:03.694705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:03.694705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:03.694705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:03.694705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:03.694705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:03.694705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:03.694705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:03:03.694705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 593/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:03:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:03.699510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc178cc-53dd-44a1-8542-cad1ec131247 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:03.759200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:03.798018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc178cc-53dd-44a1-8542-cad1ec131247 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:03.833696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d9378-7a01-4594-9be2-eeb02eda3b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:03.834303 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d9378-7a01-4594-9be2-eeb02eda3b45 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:03.834579 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d9378-7a01-4594-9be2-eeb02eda3b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:03.834873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d9378-7a01-4594-9be2-eeb02eda3b45 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:03.835534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d9378-7a01-4594-9be2-eeb02eda3b45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:03.835833 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d9378-7a01-4594-9be2-eeb02eda3b45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:03.837195 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d9378-7a01-4594-9be2-eeb02eda3b45 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:03.845074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 588/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:03.851057 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a83028-0e7f-4f8a-b32d-ee6c6ef09b99 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:03.953345 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a83028-0e7f-4f8a-b32d-ee6c6ef09b99 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:03.988019 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37363d-1ff4-4139-a129-8889580e90c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:03.988776 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37363d-1ff4-4139-a129-8889580e90c6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:03.989062 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37363d-1ff4-4139-a129-8889580e90c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:03.989335 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37363d-1ff4-4139-a129-8889580e90c6 None admin] PATH_INFO: `/v3/users/acfa4089a4d849e3a7357f783aa618c6` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:04.002282 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b37363d-1ff4-4139-a129-8889580e90c6 None admin] RBAC: Authorizing `identity:get_user(user_id=acfa4089a4d849e3a7357f783aa618c6)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:04.003773 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b37363d-1ff4-4139-a129-8889580e90c6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:04.005496 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 594/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:03:03 2022] GET /identity/v3/users/acfa4089a4d849e3a7357f783aa618c6 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:04.011544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d9378-7a01-4594-9be2-eeb02eda3b45 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:04.109160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d9378-7a01-4594-9be2-eeb02eda3b45 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:04.132252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:04.143254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee967fde-cbe5-45b9-95f8-7f302c9a8189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:04.143860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee967fde-cbe5-45b9-95f8-7f302c9a8189 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:04.144141 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee967fde-cbe5-45b9-95f8-7f302c9a8189 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:04.144967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee967fde-cbe5-45b9-95f8-7f302c9a8189 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:04.212450 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee967fde-cbe5-45b9-95f8-7f302c9a8189 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:04.213904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee967fde-cbe5-45b9-95f8-7f302c9a8189 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:04.312989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 589/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:03:04.319730 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37363d-1ff4-4139-a129-8889580e90c6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:04.338379 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:04.425234 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37363d-1ff4-4139-a129-8889580e90c6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:04.459403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcafcc-410a-4b70-a933-2aebd1f129c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:04.460026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcafcc-410a-4b70-a933-2aebd1f129c8 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:04.460305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcafcc-410a-4b70-a933-2aebd1f129c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:04.460577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcafcc-410a-4b70-a933-2aebd1f129c8 None admin] PATH_INFO: `/v3/projects/c34c25f6bf8043e086fcdbd5b7b4781d` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:04.461685 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbcafcc-410a-4b70-a933-2aebd1f129c8 None admin] RBAC: Authorizing `identity:get_project(project_id=c34c25f6bf8043e086fcdbd5b7b4781d)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:04.463316 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbcafcc-410a-4b70-a933-2aebd1f129c8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:04.464950 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 595/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:03:04 2022] GET /identity/v3/projects/c34c25f6bf8043e086fcdbd5b7b4781d => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:04.471191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee967fde-cbe5-45b9-95f8-7f302c9a8189 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:04.496411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:04.572946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee967fde-cbe5-45b9-95f8-7f302c9a8189 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:04.606749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208d445-2426-410b-85ce-2fd637190dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:04.607348 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208d445-2426-410b-85ce-2fd637190dbf None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:04.607625 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208d445-2426-410b-85ce-2fd637190dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:04.607892 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208d445-2426-410b-85ce-2fd637190dbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:04.608603 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208d445-2426-410b-85ce-2fd637190dbf None admin] RBAC: Adding query filter params (role.id=ba05677650e046a48d5f19440d48fbf2, scope.project.id=c34c25f6bf8043e086fcdbd5b7b4781d, user.id=acfa4089a4d849e3a7357f783aa618c6) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:04.608906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208d445-2426-410b-85ce-2fd637190dbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:04.610317 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208d445-2426-410b-85ce-2fd637190dbf None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:04.615747 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 590/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:04 2022] GET /identity/v3/role_assignments?user.id=acfa4089a4d849e3a7357f783aa618c6&scope.project.id=c34c25f6bf8043e086fcdbd5b7b4781d&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:06.527593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c76047-6b39-4b02-a2a3-bb180f772e57 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:06.528162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c76047-6b39-4b02-a2a3-bb180f772e57 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:06.528162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c76047-6b39-4b02-a2a3-bb180f772e57 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:06.529488 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 596/1186] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:06.534731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5dccfa-82a7-4b6e-9aef-d6e4da39a571 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:06.535155 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5dccfa-82a7-4b6e-9aef-d6e4da39a571 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:06.535315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5dccfa-82a7-4b6e-9aef-d6e4da39a571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:06.554914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1b5dccfa-82a7-4b6e-9aef-d6e4da39a571 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:06.556108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1b5dccfa-82a7-4b6e-9aef-d6e4da39a571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:06.622262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 591/1187] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:06.914128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c76047-6b39-4b02-a2a3-bb180f772e57 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:06.999878 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:07.017078 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c76047-6b39-4b02-a2a3-bb180f772e57 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:07.051412 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385be8c0-417d-4795-9bf0-cfadde178d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:07.052029 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385be8c0-417d-4795-9bf0-cfadde178d72 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:07.052552 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385be8c0-417d-4795-9bf0-cfadde178d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:07.052832 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385be8c0-417d-4795-9bf0-cfadde178d72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:07.062863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385be8c0-417d-4795-9bf0-cfadde178d72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:07.067838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-385be8c0-417d-4795-9bf0-cfadde178d72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:07.067838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:07.067838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:07.067838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:07.067838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:07.067838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:07.067838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:07.067838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:07.067838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:07.067838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:07.067838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:07.067838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:07.067838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:07.067838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:07.067838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:07.067838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:07.068930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:07.068930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:07.068930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:07.068930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:07.068930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:07.068930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:07.068930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:07.068930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:07.068930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:07.068930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:07.068930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:07.068930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:07.068930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:07.068930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:07.068930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:07.068930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:07.069775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:07.069775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:07.069775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:07.069775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:07.069775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:07.069775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:07.069775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:07.069775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:07.069775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:07.069775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:07.069775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:07.069775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:03:07.069775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 597/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:03:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:07.075030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5dccfa-82a7-4b6e-9aef-d6e4da39a571 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:07.094420 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:07.179148 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5dccfa-82a7-4b6e-9aef-d6e4da39a571 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:07.213784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada10ba8-6b6a-4b35-af18-997c7c8aae81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:07.214394 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada10ba8-6b6a-4b35-af18-997c7c8aae81 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:07.214702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada10ba8-6b6a-4b35-af18-997c7c8aae81 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:07.214977 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada10ba8-6b6a-4b35-af18-997c7c8aae81 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:07.215638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada10ba8-6b6a-4b35-af18-997c7c8aae81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:07.215943 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada10ba8-6b6a-4b35-af18-997c7c8aae81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:07.217318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada10ba8-6b6a-4b35-af18-997c7c8aae81 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:07.225589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 592/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:07.231669 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385be8c0-417d-4795-9bf0-cfadde178d72 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:07.334185 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385be8c0-417d-4795-9bf0-cfadde178d72 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:07.369318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e14e19-e6d5-4e9c-9b25-cf2c7b3abcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:07.369937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e14e19-e6d5-4e9c-9b25-cf2c7b3abcbd None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:07.370214 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e14e19-e6d5-4e9c-9b25-cf2c7b3abcbd None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:07.370485 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e14e19-e6d5-4e9c-9b25-cf2c7b3abcbd None admin] PATH_INFO: `/v3/users/acfa4089a4d849e3a7357f783aa618c6` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:07.371636 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e14e19-e6d5-4e9c-9b25-cf2c7b3abcbd None admin] RBAC: Authorizing `identity:get_user(user_id=acfa4089a4d849e3a7357f783aa618c6)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:07.373069 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e14e19-e6d5-4e9c-9b25-cf2c7b3abcbd None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:07.374654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 598/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:03:07 2022] GET /identity/v3/users/acfa4089a4d849e3a7357f783aa618c6 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:07.381392 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada10ba8-6b6a-4b35-af18-997c7c8aae81 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:07.467146 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:07.486077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada10ba8-6b6a-4b35-af18-997c7c8aae81 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:07.519487 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329474e9-0da1-43f4-b6f7-7ab9567d5835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:07.520099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329474e9-0da1-43f4-b6f7-7ab9567d5835 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:07.520373 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329474e9-0da1-43f4-b6f7-7ab9567d5835 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:07.520641 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329474e9-0da1-43f4-b6f7-7ab9567d5835 None admin] PATH_INFO: `/v3/projects/c34c25f6bf8043e086fcdbd5b7b4781d` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:07.521724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329474e9-0da1-43f4-b6f7-7ab9567d5835 None admin] RBAC: Authorizing `identity:get_project(project_id=c34c25f6bf8043e086fcdbd5b7b4781d)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:07.523174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329474e9-0da1-43f4-b6f7-7ab9567d5835 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:07.524774 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 593/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:03:07 2022] GET /identity/v3/projects/c34c25f6bf8043e086fcdbd5b7b4781d => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:07.530727 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e14e19-e6d5-4e9c-9b25-cf2c7b3abcbd None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:07.545392 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:07.629189 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e14e19-e6d5-4e9c-9b25-cf2c7b3abcbd None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:07.661402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6940d87-d4ed-41d4-829d-1cc1bc511a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:07.662032 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6940d87-d4ed-41d4-829d-1cc1bc511a46 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:07.662307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6940d87-d4ed-41d4-829d-1cc1bc511a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:07.662593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6940d87-d4ed-41d4-829d-1cc1bc511a46 None admin] PATH_INFO: `/v3/projects/c34c25f6bf8043e086fcdbd5b7b4781d/users/acfa4089a4d849e3a7357f783aa618c6/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:07.664492 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6940d87-d4ed-41d4-829d-1cc1bc511a46 None admin] RBAC: Authorizing `identity:create_grant(project_id=c34c25f6bf8043e086fcdbd5b7b4781d, user_id=acfa4089a4d849e3a7357f783aa618c6, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:07.665900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6940d87-d4ed-41d4-829d-1cc1bc511a46 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:07.678671 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 599/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:03:07 2022] PUT /identity/v3/projects/c34c25f6bf8043e086fcdbd5b7b4781d/users/acfa4089a4d849e3a7357f783aa618c6/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:03:09.567608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a7aa3-17a8-4d7d-a050-5c2d14c98775 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:09.568130 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a7aa3-17a8-4d7d-a050-5c2d14c98775 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:09.568130 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a7aa3-17a8-4d7d-a050-5c2d14c98775 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:09.569465 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 594/1193] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:03:09.574766 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84537533-2233-4bd2-90b7-416b353c88a0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:09.575203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84537533-2233-4bd2-90b7-416b353c88a0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:09.575321 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84537533-2233-4bd2-90b7-416b353c88a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:09.594177 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-84537533-2233-4bd2-90b7-416b353c88a0 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:03:09.595411 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-84537533-2233-4bd2-90b7-416b353c88a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:03:09.665269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 600/1194] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:03:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:03:09.941703 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a7aa3-17a8-4d7d-a050-5c2d14c98775 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:10.064390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a7aa3-17a8-4d7d-a050-5c2d14c98775 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:10.108760 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4899ab-25ea-4ceb-870f-b112524e06bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:10.109390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4899ab-25ea-4ceb-870f-b112524e06bd None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:10.109667 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4899ab-25ea-4ceb-870f-b112524e06bd None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:10.109938 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4899ab-25ea-4ceb-870f-b112524e06bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:10.120582 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4899ab-25ea-4ceb-870f-b112524e06bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:03:10.125463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-8c4899ab-25ea-4ceb-870f-b112524e06bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:10.125463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:03:10.125463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:03:10.125463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:03:10.125463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:03:10.125463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:03:10.125463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:03:10.125463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:03:10.125463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:03:10.125463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:03:10.125463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:03:10.125463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:03:10.125463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:03:10.125463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:03:10.125463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:03:10.125463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:03:10.126560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:03:10.126560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:03:10.126560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:03:10.126560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:03:10.126560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:03:10.126560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:03:10.126560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:03:10.126560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:03:10.126560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:03:10.126560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:03:10.126560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:03:10.126560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:03:10.126560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:03:10.126560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:03:10.126560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:03:10.126560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:03:10.127481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:03:10.127481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:03:10.127481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:03:10.127481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:03:10.127481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:03:10.127481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:03:10.127481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:03:10.127481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:03:10.127481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:03:10.127481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:03:10.127481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:03:10.127481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:03:10.127481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 595/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:03:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:03:10.134063 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84537533-2233-4bd2-90b7-416b353c88a0 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:10.136318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:10.239373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84537533-2233-4bd2-90b7-416b353c88a0 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:10.274544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287b310-b6bd-4981-b7a6-2d39658c338f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:10.275151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287b310-b6bd-4981-b7a6-2d39658c338f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:10.275429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287b310-b6bd-4981-b7a6-2d39658c338f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:10.275698 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287b310-b6bd-4981-b7a6-2d39658c338f None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:10.276360 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287b310-b6bd-4981-b7a6-2d39658c338f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:10.276658 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287b310-b6bd-4981-b7a6-2d39658c338f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:10.278004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287b310-b6bd-4981-b7a6-2d39658c338f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:10.287623 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 601/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:03:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:10.293458 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4899ab-25ea-4ceb-870f-b112524e06bd None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:10.305655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:10.394652 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4899ab-25ea-4ceb-870f-b112524e06bd None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:10.428226 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32adfe-89a3-4b08-9a40-6600b0fd2ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:10.428858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32adfe-89a3-4b08-9a40-6600b0fd2ddd None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:10.429137 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32adfe-89a3-4b08-9a40-6600b0fd2ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:10.429409 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32adfe-89a3-4b08-9a40-6600b0fd2ddd None admin] PATH_INFO: `/v3/users/acfa4089a4d849e3a7357f783aa618c6` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:10.430527 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32adfe-89a3-4b08-9a40-6600b0fd2ddd None admin] RBAC: Authorizing `identity:get_user(user_id=acfa4089a4d849e3a7357f783aa618c6)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:10.432005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32adfe-89a3-4b08-9a40-6600b0fd2ddd None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:10.433582 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 596/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 6 16:03:10 2022] GET /identity/v3/users/acfa4089a4d849e3a7357f783aa618c6 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:10.439607 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287b310-b6bd-4981-b7a6-2d39658c338f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:10.502397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:10.540897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287b310-b6bd-4981-b7a6-2d39658c338f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:10.575899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea970b4-719d-43c1-9b24-60abb9d30376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:10.576534 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea970b4-719d-43c1-9b24-60abb9d30376 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:10.576767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea970b4-719d-43c1-9b24-60abb9d30376 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:10.577044 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea970b4-719d-43c1-9b24-60abb9d30376 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:10.645014 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea970b4-719d-43c1-9b24-60abb9d30376 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:10.646469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea970b4-719d-43c1-9b24-60abb9d30376 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:10.707639 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:10.751225 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 602/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:03:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:03:10.758119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32adfe-89a3-4b08-9a40-6600b0fd2ddd None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:10.837642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:10.862231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32adfe-89a3-4b08-9a40-6600b0fd2ddd None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:10.897870 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163447a2-e820-4f19-a0b8-b9f0af181764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:10.898500 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163447a2-e820-4f19-a0b8-b9f0af181764 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:10.898805 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163447a2-e820-4f19-a0b8-b9f0af181764 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:10.899077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163447a2-e820-4f19-a0b8-b9f0af181764 None admin] PATH_INFO: `/v3/projects/c34c25f6bf8043e086fcdbd5b7b4781d` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:10.900212 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163447a2-e820-4f19-a0b8-b9f0af181764 None admin] RBAC: Authorizing `identity:get_project(project_id=c34c25f6bf8043e086fcdbd5b7b4781d)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:10.901644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163447a2-e820-4f19-a0b8-b9f0af181764 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:10.903299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 597/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 6 16:03:10 2022] GET /identity/v3/projects/c34c25f6bf8043e086fcdbd5b7b4781d => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:03:10.909430 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea970b4-719d-43c1-9b24-60abb9d30376 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:03:11.012897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea970b4-719d-43c1-9b24-60abb9d30376 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:03:11.049080 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f67f46-c680-4ab5-971d-e5263d688ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:03:11.049666 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f67f46-c680-4ab5-971d-e5263d688ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:03:11.049941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f67f46-c680-4ab5-971d-e5263d688ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:03:11.050205 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f67f46-c680-4ab5-971d-e5263d688ba9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:03:11.051151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f67f46-c680-4ab5-971d-e5263d688ba9 None admin] RBAC: Adding query filter params (role.id=ba05677650e046a48d5f19440d48fbf2, scope.project.id=c34c25f6bf8043e086fcdbd5b7b4781d, user.id=acfa4089a4d849e3a7357f783aa618c6) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:03:11.051324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f67f46-c680-4ab5-971d-e5263d688ba9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:03:11.052987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f67f46-c680-4ab5-971d-e5263d688ba9 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:03:11.059003 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 603/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:03:10 2022] GET /identity/v3/role_assignments?user.id=acfa4089a4d849e3a7357f783aa618c6&scope.project.id=c34c25f6bf8043e086fcdbd5b7b4781d&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:03:20.893005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=28 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:03:21.056125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=86 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:09.573579 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a3df4-626d-4edd-9493-7f863033aa27 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:09.574553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a3df4-626d-4edd-9493-7f863033aa27 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:09.574553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a3df4-626d-4edd-9493-7f863033aa27 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:09.576301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 598/1201] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:04:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:09.586794 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3408d-b465-477a-be32-aa5ef7ee736f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:09.587506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3408d-b465-477a-be32-aa5ef7ee736f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:09.587506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3408d-b465-477a-be32-aa5ef7ee736f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:09.617843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7cb3408d-b465-477a-be32-aa5ef7ee736f None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:09.620012 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7cb3408d-b465-477a-be32-aa5ef7ee736f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:09.750682 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 604/1202] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:04:10.199583 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a3df4-626d-4edd-9493-7f863033aa27 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:10.398207 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a3df4-626d-4edd-9493-7f863033aa27 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:10.481164 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21739edc-3f4d-4e9f-9569-059c342544f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:10.482726 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21739edc-3f4d-4e9f-9569-059c342544f8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:10.483297 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21739edc-3f4d-4e9f-9569-059c342544f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:10.483853 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21739edc-3f4d-4e9f-9569-059c342544f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:10.510514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21739edc-3f4d-4e9f-9569-059c342544f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:10.512958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21739edc-3f4d-4e9f-9569-059c342544f8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:10.549639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-21739edc-3f4d-4e9f-9569-059c342544f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:10.549639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:10.549639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:10.549639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:10.549639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:10.549639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:10.549639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:10.549639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:10.549639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:10.549639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:10.549639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:10.549639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:10.549639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:10.549639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:10.549639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:10.549639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:10.551488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:10.551488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:10.551488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:10.551488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:10.551488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:10.551488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:10.551488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:10.551488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:10.551488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:10.551488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:10.551488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:10.551488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:10.551488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:10.551488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:10.551488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:10.551488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:10.551488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:10.552672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:10.552672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:10.552672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:04:10.557803 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 599/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:10.566332 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3408d-b465-477a-be32-aa5ef7ee736f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:10.761307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3408d-b465-477a-be32-aa5ef7ee736f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:10.809854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:10.818873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a6acb-09da-4d49-ab09-d8a9b40d30fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:10.819573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a6acb-09da-4d49-ab09-d8a9b40d30fc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:10.819998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a6acb-09da-4d49-ab09-d8a9b40d30fc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:10.820550 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a6acb-09da-4d49-ab09-d8a9b40d30fc None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:10.824304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a6acb-09da-4d49-ab09-d8a9b40d30fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:10.824775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a6acb-09da-4d49-ab09-d8a9b40d30fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:10.826432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a6acb-09da-4d49-ab09-d8a9b40d30fc None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:10.851300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 605/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:10.855779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21739edc-3f4d-4e9f-9569-059c342544f8 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:10.949247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:11.044882 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21739edc-3f4d-4e9f-9569-059c342544f8 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:11.141745 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c9331-53c1-4adb-a93f-8845f1351dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:11.142479 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c9331-53c1-4adb-a93f-8845f1351dab None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:11.142762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c9331-53c1-4adb-a93f-8845f1351dab None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:11.143170 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c9331-53c1-4adb-a93f-8845f1351dab None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:11.143935 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c9331-53c1-4adb-a93f-8845f1351dab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:11.145748 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c9331-53c1-4adb-a93f-8845f1351dab None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:11.174709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 600/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 6 16:04:10 2022] POST /identity/v3/users => generated 251 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:04:13.735334 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e91bd-bf3a-4289-94de-e78473d89c6a None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:13.736028 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e91bd-bf3a-4289-94de-e78473d89c6a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:13.736028 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e91bd-bf3a-4289-94de-e78473d89c6a None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:13.737777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 606/1206] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:04:13.743995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dcd951-302b-46dd-b773-1458fbdf4cf6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:13.744515 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dcd951-302b-46dd-b773-1458fbdf4cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:13.744515 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dcd951-302b-46dd-b773-1458fbdf4cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:13.764206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-c7dcd951-302b-46dd-b773-1458fbdf4cf6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:13.765442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c7dcd951-302b-46dd-b773-1458fbdf4cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:13.847156 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 601/1207] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:14.128652 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e91bd-bf3a-4289-94de-e78473d89c6a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:14.285033 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e91bd-bf3a-4289-94de-e78473d89c6a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:14.343078 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afd8ef-da3f-4eaa-9395-a0875dc04281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:14.343913 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afd8ef-da3f-4eaa-9395-a0875dc04281 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:14.344230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afd8ef-da3f-4eaa-9395-a0875dc04281 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:14.344515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afd8ef-da3f-4eaa-9395-a0875dc04281 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:14.362912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afd8ef-da3f-4eaa-9395-a0875dc04281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:04:14.370803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-d8afd8ef-da3f-4eaa-9395-a0875dc04281 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:04:14.370803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:14.370803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:14.370803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:14.370803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:14.370803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:14.370803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:14.370803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:14.370803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:14.370803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:14.370803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:14.370803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:14.370803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:04:14.370803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:04:14.370803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:04:14.370803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:04:14.373468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:04:14.373468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:04:14.373468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:04:14.373468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:04:14.373468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:14.373468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:14.373468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:14.373468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:14.373468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:14.373468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:14.373468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:14.373468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:14.373468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:14.373468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:14.373468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:14.373468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:14.376464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:14.376464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:14.376464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:14.376464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:14.376464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:04:14.376464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:04:14.376464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:04:14.376464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:04:14.376464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:04:14.376464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:04:14.376464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:04:14.376464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:14.376464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 607/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:04:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:14.380080 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dcd951-302b-46dd-b773-1458fbdf4cf6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:14.436482 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:14.514965 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dcd951-302b-46dd-b773-1458fbdf4cf6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:14.556209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ee2f7-507a-4cd8-9559-01ffab49b9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:14.557056 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ee2f7-507a-4cd8-9559-01ffab49b9fc None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:14.557428 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ee2f7-507a-4cd8-9559-01ffab49b9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:14.557800 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ee2f7-507a-4cd8-9559-01ffab49b9fc None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:14.558575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ee2f7-507a-4cd8-9559-01ffab49b9fc None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:14.559030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ee2f7-507a-4cd8-9559-01ffab49b9fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:14.560776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ee2f7-507a-4cd8-9559-01ffab49b9fc None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:14.571705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 602/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:04:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:14.579980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afd8ef-da3f-4eaa-9395-a0875dc04281 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:14.648636 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:14.720768 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afd8ef-da3f-4eaa-9395-a0875dc04281 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:14.763524 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d28b9-1b56-4238-9567-00be2f3f0afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:14.765897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d28b9-1b56-4238-9567-00be2f3f0afd None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:14.765897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d28b9-1b56-4238-9567-00be2f3f0afd None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:14.765897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d28b9-1b56-4238-9567-00be2f3f0afd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:14.777080 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d28b9-1b56-4238-9567-00be2f3f0afd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:14.778537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d28b9-1b56-4238-9567-00be2f3f0afd None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:14.796058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-921d28b9-1b56-4238-9567-00be2f3f0afd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:14.796058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:14.796058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:14.796058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:14.796058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:14.796058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:14.796058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:14.796058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:14.796058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:14.796058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:14.796058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:14.796058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:14.796058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:14.796058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:14.796058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:14.796058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:14.797423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:14.797423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:14.797423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:14.797423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:14.797423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:14.797423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:14.797423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:14.797423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:14.797423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:14.797423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:14.797423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:14.797423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:14.797423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:14.797423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:14.797423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:14.797423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:14.797423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:14.798784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:14.798784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:14.798784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:14.798784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 608/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:14.807307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ee2f7-507a-4cd8-9559-01ffab49b9fc None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:14.950586 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:14.955665 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ee2f7-507a-4cd8-9559-01ffab49b9fc None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:15.009891 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2586aa-da82-4196-ab79-fee186bbdeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:15.010630 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2586aa-da82-4196-ab79-fee186bbdeef None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:15.010940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2586aa-da82-4196-ab79-fee186bbdeef None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:15.011212 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2586aa-da82-4196-ab79-fee186bbdeef None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:15.011874 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2586aa-da82-4196-ab79-fee186bbdeef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:15.012180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2586aa-da82-4196-ab79-fee186bbdeef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:15.013709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2586aa-da82-4196-ab79-fee186bbdeef None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:15.030847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 603/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:15.038632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d28b9-1b56-4238-9567-00be2f3f0afd None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:15.202009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d28b9-1b56-4238-9567-00be2f3f0afd None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:15.203610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:15.256014 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e150e-9dfa-4286-8729-ed083e6b31a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:15.256014 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e150e-9dfa-4286-8729-ed083e6b31a8 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:15.257358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e150e-9dfa-4286-8729-ed083e6b31a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:15.257358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e150e-9dfa-4286-8729-ed083e6b31a8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:15.267948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e150e-9dfa-4286-8729-ed083e6b31a8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:15.270968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e150e-9dfa-4286-8729-ed083e6b31a8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:15.287207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-ae9e150e-9dfa-4286-8729-ed083e6b31a8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:04:15.287207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:15.287207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:15.287207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:15.287207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:15.287207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:15.287207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:15.287207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:15.287207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:15.287207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:15.287207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:15.287207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:15.287207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:04:15.287207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:04:15.287207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:04:15.287207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:04:15.288358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:15.288358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:15.288358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:04:15.288358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:15.288358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:04:15.288358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:15.288358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:15.288358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:15.288358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:15.288358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:15.288358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:15.288358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:15.288358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:15.288358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:15.288358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:15.288358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:15.288358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:15.288358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:15.290139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:15.290139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:15.290139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:04:15.290139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:04:15.290139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:04:15.290139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:04:15.290139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:04:15.290139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:04:15.290139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:04:15.290139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:15.290139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 609/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:15.297868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2586aa-da82-4196-ab79-fee186bbdeef None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:15.449692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2586aa-da82-4196-ab79-fee186bbdeef None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:15.506020 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ba62f-33cf-4452-9859-4417ff54efbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:15.507088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ba62f-33cf-4452-9859-4417ff54efbd None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:15.507356 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ba62f-33cf-4452-9859-4417ff54efbd None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:15.507702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ba62f-33cf-4452-9859-4417ff54efbd None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:15.508621 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ba62f-33cf-4452-9859-4417ff54efbd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:15.509130 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ba62f-33cf-4452-9859-4417ff54efbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:15.511204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ba62f-33cf-4452-9859-4417ff54efbd None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:15.540450 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 604/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Sep 6 16:04:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:15.548255 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e150e-9dfa-4286-8729-ed083e6b31a8 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:15.716404 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e150e-9dfa-4286-8729-ed083e6b31a8 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:15.772598 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866baa9c-3701-41e0-93b6-f3b76cc2c053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:15.773210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866baa9c-3701-41e0-93b6-f3b76cc2c053 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:15.773377 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866baa9c-3701-41e0-93b6-f3b76cc2c053 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:15.773710 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866baa9c-3701-41e0-93b6-f3b76cc2c053 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:15.822853 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:15.911520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866baa9c-3701-41e0-93b6-f3b76cc2c053 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:15.912463 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866baa9c-3701-41e0-93b6-f3b76cc2c053 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:16.070009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 610/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:04:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:04:16.076209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ba62f-33cf-4452-9859-4417ff54efbd None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:16.103957 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:16.222361 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ba62f-33cf-4452-9859-4417ff54efbd None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:16.265355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd20059-d729-49b3-9675-7ecad7b3af5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:16.267075 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd20059-d729-49b3-9675-7ecad7b3af5e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:16.267075 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd20059-d729-49b3-9675-7ecad7b3af5e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:16.267075 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd20059-d729-49b3-9675-7ecad7b3af5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:16.281670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd20059-d729-49b3-9675-7ecad7b3af5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:16.284041 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd20059-d729-49b3-9675-7ecad7b3af5e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:16.316355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-9dd20059-d729-49b3-9675-7ecad7b3af5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:16.316355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:16.316355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:16.316355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:16.316355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:16.316355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:16.316355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:16.316355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:16.316355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:16.316355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:16.316355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:16.316355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:16.316355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:16.316355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:16.316355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:16.316355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:16.319010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:16.319010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:16.319010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:16.319010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:16.319010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:16.319010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:16.319010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:16.319010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:16.319010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:16.319010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:16.319010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:16.319010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:16.319010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:16.319010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:16.319010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:16.319010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:16.319010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:16.320141 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:16.320141 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:16.320141 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:04:16.320141 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 605/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:16.324781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866baa9c-3701-41e0-93b6-f3b76cc2c053 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:16.426828 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:16.499420 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866baa9c-3701-41e0-93b6-f3b76cc2c053 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:16.548659 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f7821-b5d7-469c-8f75-7b92dcf58f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:16.549352 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f7821-b5d7-469c-8f75-7b92dcf58f33 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:16.549711 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f7821-b5d7-469c-8f75-7b92dcf58f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:16.550012 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f7821-b5d7-469c-8f75-7b92dcf58f33 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:16.550899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f7821-b5d7-469c-8f75-7b92dcf58f33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:16.551162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f7821-b5d7-469c-8f75-7b92dcf58f33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:16.552827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f7821-b5d7-469c-8f75-7b92dcf58f33 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:16.571921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 611/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:16.582045 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd20059-d729-49b3-9675-7ecad7b3af5e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:16.681889 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:16.734115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd20059-d729-49b3-9675-7ecad7b3af5e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:16.790321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d8fe7-952c-493e-b779-263ab69039d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:16.791150 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d8fe7-952c-493e-b779-263ab69039d2 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:16.791536 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d8fe7-952c-493e-b779-263ab69039d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:16.791970 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d8fe7-952c-493e-b779-263ab69039d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:16.801250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d8fe7-952c-493e-b779-263ab69039d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:16.803049 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d8fe7-952c-493e-b779-263ab69039d2 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:16.817544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-9a6d8fe7-952c-493e-b779-263ab69039d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:16.817544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:16.817544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:16.817544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:16.817544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:16.817544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:16.817544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:16.817544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:16.817544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:16.817544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:16.817544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:16.817544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:16.817544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:04:16.817544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:04:16.817544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:04:16.819587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:04:16.819587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:16.819587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:16.819587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:16.819587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:16.819587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:16.819587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:16.819587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:16.819587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:16.819587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:16.819587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:16.819587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:16.819587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:16.819587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:16.819587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:16.819587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:16.819587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:16.821615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:04:16.821615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:04:16.821615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:04:16.821615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:04:16.821615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:04:16.821615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:04:16.821615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:16.821615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:04:16.825499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 606/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:04:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:16.827911 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f7821-b5d7-469c-8f75-7b92dcf58f33 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:16.958163 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f7821-b5d7-469c-8f75-7b92dcf58f33 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:16.991696 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:17.014432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662e384-4c5b-40ee-aa3f-50a87509ee75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:17.015348 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662e384-4c5b-40ee-aa3f-50a87509ee75 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:17.015790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662e384-4c5b-40ee-aa3f-50a87509ee75 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:17.016162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662e384-4c5b-40ee-aa3f-50a87509ee75 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:17.017055 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662e384-4c5b-40ee-aa3f-50a87509ee75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:17.017476 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662e384-4c5b-40ee-aa3f-50a87509ee75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:17.019971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662e384-4c5b-40ee-aa3f-50a87509ee75 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:17.039064 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 612/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:04:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:17.047454 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d8fe7-952c-493e-b779-263ab69039d2 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:17.213737 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d8fe7-952c-493e-b779-263ab69039d2 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:17.265390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:17.276670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d24bf-b28b-456d-bbb8-8209e2899765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:17.277736 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d24bf-b28b-456d-bbb8-8209e2899765 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:17.278230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d24bf-b28b-456d-bbb8-8209e2899765 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:17.278780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d24bf-b28b-456d-bbb8-8209e2899765 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:17.279738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d24bf-b28b-456d-bbb8-8209e2899765 None admin] RBAC: Adding query filter params (role.id=56d621cf28924933b6356795a6f8ba23, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=034d75ab747e4c0cab040e47c6a6236e) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:17.280187 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d24bf-b28b-456d-bbb8-8209e2899765 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:17.281916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d24bf-b28b-456d-bbb8-8209e2899765 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:17.290416 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 607/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:04:17 2022] GET /identity/v3/role_assignments?user.id=034d75ab747e4c0cab040e47c6a6236e&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=56d621cf28924933b6356795a6f8ba23 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:19.658044 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d143a57-8e3f-4310-ba43-079c1e50219b None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:19.658990 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d143a57-8e3f-4310-ba43-079c1e50219b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:19.658990 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d143a57-8e3f-4310-ba43-079c1e50219b None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:19.662100 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 613/1220] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:04:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:19.669482 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c6048-0e3d-4b41-b9b6-853e951b394d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:19.670120 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c6048-0e3d-4b41-b9b6-853e951b394d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:19.670571 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c6048-0e3d-4b41-b9b6-853e951b394d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:19.699655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-6a1c6048-0e3d-4b41-b9b6-853e951b394d None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:19.701360 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6a1c6048-0e3d-4b41-b9b6-853e951b394d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:19.796359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 608/1221] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:04:20.108716 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d143a57-8e3f-4310-ba43-079c1e50219b None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:20.271455 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d143a57-8e3f-4310-ba43-079c1e50219b None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:20.332761 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5021ef-ee3b-4da7-87f8-a99f76d4660d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:20.333482 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5021ef-ee3b-4da7-87f8-a99f76d4660d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:20.333855 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5021ef-ee3b-4da7-87f8-a99f76d4660d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:20.334230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5021ef-ee3b-4da7-87f8-a99f76d4660d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:20.352261 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5021ef-ee3b-4da7-87f8-a99f76d4660d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:04:20.361883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-2c5021ef-ee3b-4da7-87f8-a99f76d4660d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:04:20.361883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:20.361883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:20.361883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:20.361883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:20.361883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:20.361883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:20.361883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:20.361883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:20.361883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:20.361883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:20.361883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:20.361883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:04:20.361883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:04:20.361883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:04:20.361883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:04:20.363125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:04:20.363125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:04:20.363125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:04:20.363125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:04:20.363125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:20.363125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:20.363125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:20.363125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:20.363125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:20.363125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:20.363125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:20.363125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:20.363125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:20.363125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:20.363125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:20.363125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:20.364147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:20.364147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:20.364147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:20.364147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:20.364147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:04:20.364147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:04:20.364147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:04:20.364147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:04:20.364147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:04:20.364147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:04:20.364147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:04:20.364147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:20.364147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 614/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:04:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:20.373322 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c6048-0e3d-4b41-b9b6-853e951b394d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:20.524250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c6048-0e3d-4b41-b9b6-853e951b394d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:20.572022 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:20.588119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e8d07-f56a-48a1-a846-0112392f3aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:20.589151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e8d07-f56a-48a1-a846-0112392f3aeb None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:20.589640 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e8d07-f56a-48a1-a846-0112392f3aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:20.590103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e8d07-f56a-48a1-a846-0112392f3aeb None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:20.591248 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e8d07-f56a-48a1-a846-0112392f3aeb None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:20.591637 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e8d07-f56a-48a1-a846-0112392f3aeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:20.593111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e8d07-f56a-48a1-a846-0112392f3aeb None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:20.603488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 609/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:04:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:20.610830 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5021ef-ee3b-4da7-87f8-a99f76d4660d None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:20.669608 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:20.761350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5021ef-ee3b-4da7-87f8-a99f76d4660d None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:20.809104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ad822-359e-4e1f-8ad6-5e51d180b318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:20.809728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ad822-359e-4e1f-8ad6-5e51d180b318 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:20.810000 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ad822-359e-4e1f-8ad6-5e51d180b318 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:20.810271 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ad822-359e-4e1f-8ad6-5e51d180b318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:20.820956 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ad822-359e-4e1f-8ad6-5e51d180b318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:20.822384 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ad822-359e-4e1f-8ad6-5e51d180b318 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:20.840587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-027ad822-359e-4e1f-8ad6-5e51d180b318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:20.840587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:20.840587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:20.840587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:20.840587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:20.840587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:20.840587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:20.840587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:20.840587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:20.840587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:20.840587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:20.840587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:20.840587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:20.840587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:20.840587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:20.840587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:20.841895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:20.841895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:20.841895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:20.841895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:20.841895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:20.841895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:20.841895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:20.841895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:20.841895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:20.841895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:20.841895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:20.841895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:20.841895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:20.841895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:20.841895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:20.841895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:20.841895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:20.842905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:20.842905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:20.842905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:20.842905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 615/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:20.851276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e8d07-f56a-48a1-a846-0112392f3aeb None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:20.985190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e8d07-f56a-48a1-a846-0112392f3aeb None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:21.026549 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db486a-fac5-471e-916d-1caec9a34499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:21.027200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db486a-fac5-471e-916d-1caec9a34499 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:21.027471 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db486a-fac5-471e-916d-1caec9a34499 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:21.027742 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db486a-fac5-471e-916d-1caec9a34499 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:21.028486 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db486a-fac5-471e-916d-1caec9a34499 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:21.028789 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db486a-fac5-471e-916d-1caec9a34499 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:21.030182 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db486a-fac5-471e-916d-1caec9a34499 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:21.044615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 610/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:21.051042 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ad822-359e-4e1f-8ad6-5e51d180b318 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:21.170937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:21.182282 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ad822-359e-4e1f-8ad6-5e51d180b318 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:21.231673 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677fdce-a29a-44d7-a1f9-830f71fb5135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:21.232403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677fdce-a29a-44d7-a1f9-830f71fb5135 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:21.232714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677fdce-a29a-44d7-a1f9-830f71fb5135 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:21.232991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677fdce-a29a-44d7-a1f9-830f71fb5135 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:21.242799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677fdce-a29a-44d7-a1f9-830f71fb5135 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:21.244811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677fdce-a29a-44d7-a1f9-830f71fb5135 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:21.261726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-4677fdce-a29a-44d7-a1f9-830f71fb5135 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:04:21.261726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:21.261726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:21.261726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:21.261726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:21.261726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:21.261726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:21.261726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:21.261726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:21.261726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:21.261726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:21.261726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:21.261726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:04:21.261726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:04:21.261726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:04:21.261726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:04:21.263004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:21.263004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:21.263004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:04:21.263004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:21.263004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:04:21.263004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:21.263004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:21.263004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:21.263004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:21.263004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:21.263004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:21.263004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:21.263004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:21.263004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:21.263004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:21.263004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:21.263004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:21.263004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:21.264143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:21.264143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:21.264143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:04:21.264143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:04:21.264143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:04:21.264143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:04:21.264143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:04:21.264143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:04:21.264143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:04:21.264143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:21.264143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 616/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:21.270325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db486a-fac5-471e-916d-1caec9a34499 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:21.337985 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:21.429914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db486a-fac5-471e-916d-1caec9a34499 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:21.475737 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3981e2-bfe3-4af7-9045-120054130d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:21.476816 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3981e2-bfe3-4af7-9045-120054130d69 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:21.477284 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3981e2-bfe3-4af7-9045-120054130d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:21.477744 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3981e2-bfe3-4af7-9045-120054130d69 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:21.479070 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3981e2-bfe3-4af7-9045-120054130d69 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:21.479592 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3981e2-bfe3-4af7-9045-120054130d69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:21.482207 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3981e2-bfe3-4af7-9045-120054130d69 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:21.513584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 611/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Sep 6 16:04:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:21.520218 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677fdce-a29a-44d7-a1f9-830f71fb5135 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:21.648520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677fdce-a29a-44d7-a1f9-830f71fb5135 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:21.697835 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d00197-1153-431e-abef-e3806d740c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:21.698769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d00197-1153-431e-abef-e3806d740c38 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:21.698949 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d00197-1153-431e-abef-e3806d740c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:21.699240 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d00197-1153-431e-abef-e3806d740c38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:21.714811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d00197-1153-431e-abef-e3806d740c38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:21.716670 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d00197-1153-431e-abef-e3806d740c38 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:21.732709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-a9d00197-1153-431e-abef-e3806d740c38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:21.732709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:21.732709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:21.732709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:21.732709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:21.732709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:21.732709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:21.732709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:21.732709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:21.732709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:21.732709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:21.732709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:21.732709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:21.732709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:21.732709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:21.732709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:21.733978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:21.733978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:21.733978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:21.733978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:21.733978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:21.733978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:21.733978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:21.733978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:21.733978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:21.733978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:21.733978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:21.733978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:21.733978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:21.733978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:21.733978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:21.733978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:21.733978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:21.735047 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:21.735047 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:21.735047 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:21.735047 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 617/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:21.743945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3981e2-bfe3-4af7-9045-120054130d69 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:21.866445 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3981e2-bfe3-4af7-9045-120054130d69 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:21.869584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:21.906907 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c958587-96a2-4b68-9939-0b234f9c5901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:21.907701 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c958587-96a2-4b68-9939-0b234f9c5901 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:21.908114 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c958587-96a2-4b68-9939-0b234f9c5901 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:21.908387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c958587-96a2-4b68-9939-0b234f9c5901 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:21.909983 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c958587-96a2-4b68-9939-0b234f9c5901 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:21.910502 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c958587-96a2-4b68-9939-0b234f9c5901 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:21.912336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c958587-96a2-4b68-9939-0b234f9c5901 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:21.927917 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 612/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:21.935395 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d00197-1153-431e-abef-e3806d740c38 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:21.949104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:22.061960 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d00197-1153-431e-abef-e3806d740c38 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:22.105947 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a737dd-e77e-4517-b07d-cf9d60acb972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:22.106576 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a737dd-e77e-4517-b07d-cf9d60acb972 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:22.106895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a737dd-e77e-4517-b07d-cf9d60acb972 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:22.107168 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a737dd-e77e-4517-b07d-cf9d60acb972 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:22.115496 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a737dd-e77e-4517-b07d-cf9d60acb972 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:22.117010 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a737dd-e77e-4517-b07d-cf9d60acb972 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:22.131158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-a1a737dd-e77e-4517-b07d-cf9d60acb972 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:22.131158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:22.131158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:22.131158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:22.131158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:22.131158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:22.131158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:22.131158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:22.131158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:22.131158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:22.131158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:22.131158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:22.131158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:04:22.131158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:04:22.131158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:04:22.132381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:04:22.132381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:22.132381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:22.132381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:22.132381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:22.132381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:22.132381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:22.132381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:22.132381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:22.132381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:22.132381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:22.132381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:22.132381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:22.132381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:22.132381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:22.132381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:22.132381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:22.133474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:04:22.133474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:04:22.133474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:04:22.133474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:04:22.133474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:04:22.133474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:04:22.133474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:22.133474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:22.133474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 618/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:04:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:22.140159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c958587-96a2-4b68-9939-0b234f9c5901 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:22.284801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c958587-96a2-4b68-9939-0b234f9c5901 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:22.332357 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff57288-227f-4880-81c6-8d87e565a172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:22.333056 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff57288-227f-4880-81c6-8d87e565a172 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:22.333344 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff57288-227f-4880-81c6-8d87e565a172 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:22.336347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff57288-227f-4880-81c6-8d87e565a172 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:22.336347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff57288-227f-4880-81c6-8d87e565a172 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:22.336347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff57288-227f-4880-81c6-8d87e565a172 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:22.336347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff57288-227f-4880-81c6-8d87e565a172 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:22.356927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 613/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:04:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:22.358321 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a737dd-e77e-4517-b07d-cf9d60acb972 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:22.450277 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:22.504044 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a737dd-e77e-4517-b07d-cf9d60acb972 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:22.550729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e8dc5-46dd-4df3-b4c1-401a8c68cbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:22.551625 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e8dc5-46dd-4df3-b4c1-401a8c68cbd2 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:22.551952 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e8dc5-46dd-4df3-b4c1-401a8c68cbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:22.552229 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e8dc5-46dd-4df3-b4c1-401a8c68cbd2 None admin] PATH_INFO: `/v3/projects/0138d62d5879495499a17e20310d4f61/users/034d75ab747e4c0cab040e47c6a6236e/roles/56d621cf28924933b6356795a6f8ba23` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:22.572750 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e8dc5-46dd-4df3-b4c1-401a8c68cbd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=0138d62d5879495499a17e20310d4f61, user_id=034d75ab747e4c0cab040e47c6a6236e, role_id=56d621cf28924933b6356795a6f8ba23)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:22.575100 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e8dc5-46dd-4df3-b4c1-401a8c68cbd2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:22.593166 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 619/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:04:22 2022] PUT /identity/v3/projects/0138d62d5879495499a17e20310d4f61/users/034d75ab747e4c0cab040e47c6a6236e/roles/56d621cf28924933b6356795a6f8ba23 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:04:24.573246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bbae1-28d3-44d0-9056-a3901441f106 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:24.573829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bbae1-28d3-44d0-9056-a3901441f106 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:24.573829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bbae1-28d3-44d0-9056-a3901441f106 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:24.575480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 614/1233] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:24.583043 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c5ecf-c34a-4137-ab86-0af147cf39eb None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:24.583628 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c5ecf-c34a-4137-ab86-0af147cf39eb None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:24.583781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c5ecf-c34a-4137-ab86-0af147cf39eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:24.612985 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-0b0c5ecf-c34a-4137-ab86-0af147cf39eb None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:24.614579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0b0c5ecf-c34a-4137-ab86-0af147cf39eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:24.710516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 620/1234] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:24.981788 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735bbae1-28d3-44d0-9056-a3901441f106 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:25.011059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:25.107847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735bbae1-28d3-44d0-9056-a3901441f106 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:25.159784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4a0cf-934e-4436-925a-daa8e287174a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:25.160864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4a0cf-934e-4436-925a-daa8e287174a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:25.161170 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4a0cf-934e-4436-925a-daa8e287174a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:25.161554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4a0cf-934e-4436-925a-daa8e287174a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:25.182953 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4a0cf-934e-4436-925a-daa8e287174a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:04:25.188442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-c2f4a0cf-934e-4436-925a-daa8e287174a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:04:25.188442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:25.188442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:25.188442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:25.188442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:25.188442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:25.188442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:25.188442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:25.188442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:25.188442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:25.188442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:25.188442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:25.188442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:04:25.188442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:04:25.188442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:04:25.188442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:04:25.189933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:04:25.189933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:04:25.189933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:04:25.189933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:04:25.189933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:25.189933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:25.189933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:25.189933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:25.189933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:25.189933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:25.189933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:25.189933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:25.189933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:25.189933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:25.189933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:25.189933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:25.191797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:25.191797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:25.191797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:25.191797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:25.191797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:04:25.191797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:04:25.191797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:04:25.191797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:04:25.191797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:04:25.191797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:04:25.191797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 16:04:25.191797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:04:25.191797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 615/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 6 16:04:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:25.197622 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c5ecf-c34a-4137-ab86-0af147cf39eb None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:25.261506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:25.348887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c5ecf-c34a-4137-ab86-0af147cf39eb None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:25.397568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4308a-23d3-4aea-9bb6-e88922eb888a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:25.398458 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4308a-23d3-4aea-9bb6-e88922eb888a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:25.398788 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4308a-23d3-4aea-9bb6-e88922eb888a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:25.399057 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4308a-23d3-4aea-9bb6-e88922eb888a None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:25.399825 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4308a-23d3-4aea-9bb6-e88922eb888a None admin] RBAC: Adding query filter params (name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:25.400189 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4308a-23d3-4aea-9bb6-e88922eb888a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:25.401765 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4308a-23d3-4aea-9bb6-e88922eb888a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:25.413060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 621/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 6 16:04:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:25.419811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4a0cf-934e-4436-925a-daa8e287174a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:25.532174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:25.555485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4a0cf-934e-4436-925a-daa8e287174a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:25.604174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a3168-0141-4d20-ae44-b5e70407f840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:25.604868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a3168-0141-4d20-ae44-b5e70407f840 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:25.605102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a3168-0141-4d20-ae44-b5e70407f840 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:25.606029 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a3168-0141-4d20-ae44-b5e70407f840 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:25.620430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a3168-0141-4d20-ae44-b5e70407f840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:25.622426 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a3168-0141-4d20-ae44-b5e70407f840 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:25.641421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-3f1a3168-0141-4d20-ae44-b5e70407f840 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:25.641421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:25.641421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:25.641421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:25.641421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:25.641421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:25.641421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:25.641421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:25.641421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:25.641421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:25.641421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:25.641421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:25.641421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:25.641421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:25.641421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:25.641421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:25.644645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:25.644645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:25.644645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:25.644645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:25.644645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:25.644645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:25.644645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:25.644645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:25.644645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:25.644645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:25.644645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:25.644645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:25.644645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:25.644645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:25.644645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:25.644645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:25.644645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:25.646466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:25.646466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:25.646466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:04:25.646466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 616/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:25.649411 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4308a-23d3-4aea-9bb6-e88922eb888a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:25.781177 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4308a-23d3-4aea-9bb6-e88922eb888a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:25.792355 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:25.822831 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19522dcd-1b79-490d-b62e-209eca5274f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:25.823454 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19522dcd-1b79-490d-b62e-209eca5274f6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:25.823763 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19522dcd-1b79-490d-b62e-209eca5274f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:25.824034 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19522dcd-1b79-490d-b62e-209eca5274f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:25.824693 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19522dcd-1b79-490d-b62e-209eca5274f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:25.824992 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19522dcd-1b79-490d-b62e-209eca5274f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:25.826392 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19522dcd-1b79-490d-b62e-209eca5274f6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:25.846452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 622/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:25.848254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a3168-0141-4d20-ae44-b5e70407f840 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:26.012702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a3168-0141-4d20-ae44-b5e70407f840 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:26.056443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c5ea5-d70c-4f4d-b253-37dedac0754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:26.057583 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c5ea5-d70c-4f4d-b253-37dedac0754d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:26.057912 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c5ea5-d70c-4f4d-b253-37dedac0754d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:26.058324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c5ea5-d70c-4f4d-b253-37dedac0754d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:26.063419 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:26.071130 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c5ea5-d70c-4f4d-b253-37dedac0754d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:26.073179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c5ea5-d70c-4f4d-b253-37dedac0754d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:26.095606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-8c1c5ea5-d70c-4f4d-b253-37dedac0754d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:04:26.095606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:26.095606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:26.095606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:26.095606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:26.095606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:26.095606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:26.095606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:26.095606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:26.095606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:26.095606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:26.095606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:26.095606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:04:26.095606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:04:26.095606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:04:26.095606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:04:26.098491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:26.098491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:26.098491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:04:26.098491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:26.098491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:04:26.098491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:26.098491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:26.098491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:26.098491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:26.098491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:26.098491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:26.098491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:26.098491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:26.098491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:26.098491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:26.098491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:26.098491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:26.098491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:26.099595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:26.099595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:26.099595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:04:26.099595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:04:26.099595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:04:26.099595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:04:26.099595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:04:26.099595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:04:26.099595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:04:26.099595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:04:26.099595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 617/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:26.104942 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19522dcd-1b79-490d-b62e-209eca5274f6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:26.235841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19522dcd-1b79-490d-b62e-209eca5274f6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:26.284878 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650edeff-248a-48d4-aabc-8c7b21f2082e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:26.285536 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650edeff-248a-48d4-aabc-8c7b21f2082e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:26.285819 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650edeff-248a-48d4-aabc-8c7b21f2082e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:26.286094 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650edeff-248a-48d4-aabc-8c7b21f2082e None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:26.286871 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650edeff-248a-48d4-aabc-8c7b21f2082e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:26.287182 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650edeff-248a-48d4-aabc-8c7b21f2082e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:26.289114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650edeff-248a-48d4-aabc-8c7b21f2082e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:26.324660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 623/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Sep 6 16:04:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:26.333146 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c5ea5-d70c-4f4d-b253-37dedac0754d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:26.482174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c5ea5-d70c-4f4d-b253-37dedac0754d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:26.527511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaecd6b-d3c0-40f7-87cd-960c9af524f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:26.528180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaecd6b-d3c0-40f7-87cd-960c9af524f0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:26.528409 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaecd6b-d3c0-40f7-87cd-960c9af524f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:26.528670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaecd6b-d3c0-40f7-87cd-960c9af524f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:26.617634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:26.625202 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaecd6b-d3c0-40f7-87cd-960c9af524f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:26.626553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaecd6b-d3c0-40f7-87cd-960c9af524f0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:26.763233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 618/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:04:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:04:26.770875 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650edeff-248a-48d4-aabc-8c7b21f2082e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:26.805980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:26.914343 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650edeff-248a-48d4-aabc-8c7b21f2082e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:26.980465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f7a67-830f-45bf-9150-0ad4e20c3dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:26.981229 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f7a67-830f-45bf-9150-0ad4e20c3dcd None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:26.981515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f7a67-830f-45bf-9150-0ad4e20c3dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:26.981786 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f7a67-830f-45bf-9150-0ad4e20c3dcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:26.996456 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f7a67-830f-45bf-9150-0ad4e20c3dcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:26.998407 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f7a67-830f-45bf-9150-0ad4e20c3dcd None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:27.017857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-0d7f7a67-830f-45bf-9150-0ad4e20c3dcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:27.017857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:27.017857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:27.017857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:27.017857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:27.017857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:27.017857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:27.017857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:27.017857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:27.017857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:27.017857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:27.017857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:27.017857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:27.017857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:27.017857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:27.017857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:27.019151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:27.019151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:27.019151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:27.019151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:27.019151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:27.019151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:27.019151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:27.019151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:27.019151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:27.019151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:27.019151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:27.019151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:27.019151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:27.019151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:27.019151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:27.019151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:27.019151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:27.020187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:27.020187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:27.020187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:27.020187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 624/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 16:04:27.030287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaecd6b-d3c0-40f7-87cd-960c9af524f0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:27.167398 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:27.174280 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaecd6b-d3c0-40f7-87cd-960c9af524f0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:27.225408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de112333-f781-434e-94c3-19eff52ddda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:27.226140 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de112333-f781-434e-94c3-19eff52ddda4 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:27.226529 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de112333-f781-434e-94c3-19eff52ddda4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:27.226893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de112333-f781-434e-94c3-19eff52ddda4 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:27.227783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de112333-f781-434e-94c3-19eff52ddda4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:27.228195 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de112333-f781-434e-94c3-19eff52ddda4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:27.230138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de112333-f781-434e-94c3-19eff52ddda4 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:27.246337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 619/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:27.252457 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f7a67-830f-45bf-9150-0ad4e20c3dcd None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:27.388135 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:27.417064 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f7a67-830f-45bf-9150-0ad4e20c3dcd None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:27.458788 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759c7d8-8fe8-4baf-942b-48198942df1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:27.459337 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759c7d8-8fe8-4baf-942b-48198942df1a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:27.459617 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759c7d8-8fe8-4baf-942b-48198942df1a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:27.459954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759c7d8-8fe8-4baf-942b-48198942df1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:27.471904 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759c7d8-8fe8-4baf-942b-48198942df1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:27.474943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759c7d8-8fe8-4baf-942b-48198942df1a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:27.500300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-2759c7d8-8fe8-4baf-942b-48198942df1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:27.500300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:27.500300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:27.500300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:27.500300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:27.500300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:27.500300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:27.500300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:27.500300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:27.500300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:27.500300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:27.500300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:27.500300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:04:27.500300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:04:27.500300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:04:27.502272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:04:27.502272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:27.502272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:27.502272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:27.502272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:27.502272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:27.502272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:27.502272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:27.502272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:27.502272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:27.502272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:27.502272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:27.502272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:27.502272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:27.502272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:27.502272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:27.502272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:27.512150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:04:27.512150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:04:27.512150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:04:27.512150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:04:27.512150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:04:27.512150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:04:27.512150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:27.512150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:27.512150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 625/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:04:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:27.514990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de112333-f781-434e-94c3-19eff52ddda4 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:27.645244 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de112333-f781-434e-94c3-19eff52ddda4 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:27.707967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432909d5-cf98-4392-a285-9a98e603d5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:27.708671 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432909d5-cf98-4392-a285-9a98e603d5f4 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:27.709088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432909d5-cf98-4392-a285-9a98e603d5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:27.709368 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432909d5-cf98-4392-a285-9a98e603d5f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:27.710548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432909d5-cf98-4392-a285-9a98e603d5f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:27.711153 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432909d5-cf98-4392-a285-9a98e603d5f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:27.713271 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432909d5-cf98-4392-a285-9a98e603d5f4 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:27.735544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 620/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:04:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:27.742497 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759c7d8-8fe8-4baf-942b-48198942df1a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:27.878625 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759c7d8-8fe8-4baf-942b-48198942df1a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:27.916673 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4927f-f585-415a-b586-fd39b2e3c18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:27.917311 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4927f-f585-415a-b586-fd39b2e3c18d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:27.917554 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4927f-f585-415a-b586-fd39b2e3c18d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:27.917828 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4927f-f585-415a-b586-fd39b2e3c18d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:27.918553 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c4927f-f585-415a-b586-fd39b2e3c18d None admin] RBAC: Adding query filter params (role.id=56d621cf28924933b6356795a6f8ba23, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=034d75ab747e4c0cab040e47c6a6236e) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:27.918889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c4927f-f585-415a-b586-fd39b2e3c18d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:27.920338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c4927f-f585-415a-b586-fd39b2e3c18d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:27.928160 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 626/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:04:27 2022] GET /identity/v3/role_assignments?user.id=034d75ab747e4c0cab040e47c6a6236e&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=56d621cf28924933b6356795a6f8ba23 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:30.360763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c13faf-7f2f-43ae-a26b-8cec0f044cfa None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:30.361349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c13faf-7f2f-43ae-a26b-8cec0f044cfa None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:30.361464 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c13faf-7f2f-43ae-a26b-8cec0f044cfa None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:30.363145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 621/1247] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:04:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:04:30.369164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5d921-a87b-4258-a0f1-de41d20d0daa None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:30.369746 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5d921-a87b-4258-a0f1-de41d20d0daa None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:30.369746 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5d921-a87b-4258-a0f1-de41d20d0daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:30.376350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:30.394602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a9d5d921-a87b-4258-a0f1-de41d20d0daa None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:30.396197 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a9d5d921-a87b-4258-a0f1-de41d20d0daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:30.501354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 627/1248] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 16:04:30.838142 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c13faf-7f2f-43ae-a26b-8cec0f044cfa None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:30.864851 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:30.986994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c13faf-7f2f-43ae-a26b-8cec0f044cfa None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:31.029096 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ca5fc-ce5f-4f39-bfb3-7149b9180317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:31.029733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ca5fc-ce5f-4f39-bfb3-7149b9180317 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:31.030061 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ca5fc-ce5f-4f39-bfb3-7149b9180317 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:31.030329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ca5fc-ce5f-4f39-bfb3-7149b9180317 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:31.045427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ca5fc-ce5f-4f39-bfb3-7149b9180317 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:04:31.052512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-ca6ca5fc-ce5f-4f39-bfb3-7149b9180317 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:04:31.052512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:31.052512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:31.052512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:31.052512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:31.052512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:31.052512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:31.052512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:31.052512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:31.052512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:31.052512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:31.052512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:31.052512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:04:31.052512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:04:31.052512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:04:31.052512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:04:31.053644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:04:31.053644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:04:31.053644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:04:31.053644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:04:31.053644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:31.053644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:31.053644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:31.053644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:31.053644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:31.053644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:31.053644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:31.053644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:31.053644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:31.053644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:31.053644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:31.053644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:31.054905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:31.054905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:31.054905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:31.054905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:31.054905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:04:31.054905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:04:31.054905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:04:31.054905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:04:31.054905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:04:31.054905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:04:31.054905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:04:31.054905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:04:31.054905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 622/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:04:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:31.061360 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5d921-a87b-4258-a0f1-de41d20d0daa None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:31.201350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5d921-a87b-4258-a0f1-de41d20d0daa None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:31.236939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:31.244539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec9d43-d482-479e-85b4-ea4fb8b0805a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:31.245368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec9d43-d482-479e-85b4-ea4fb8b0805a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:31.245884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec9d43-d482-479e-85b4-ea4fb8b0805a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:31.246262 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec9d43-d482-479e-85b4-ea4fb8b0805a None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:31.247419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec9d43-d482-479e-85b4-ea4fb8b0805a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:31.247805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec9d43-d482-479e-85b4-ea4fb8b0805a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:31.249339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec9d43-d482-479e-85b4-ea4fb8b0805a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:31.261385 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 628/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:04:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:04:31.268525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ca5fc-ce5f-4f39-bfb3-7149b9180317 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:31.396044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:31.430198 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ca5fc-ce5f-4f39-bfb3-7149b9180317 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:31.479189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25df381-7459-403f-9515-d7641781f262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:31.479739 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25df381-7459-403f-9515-d7641781f262 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:31.480005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25df381-7459-403f-9515-d7641781f262 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:31.480272 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25df381-7459-403f-9515-d7641781f262 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:31.492596 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25df381-7459-403f-9515-d7641781f262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:31.494115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25df381-7459-403f-9515-d7641781f262 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:31.516213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-a25df381-7459-403f-9515-d7641781f262 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:31.516213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:31.516213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:31.516213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:31.516213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:31.516213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:31.516213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:31.516213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:31.516213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:31.516213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:31.516213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:31.516213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:31.516213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:31.516213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:31.516213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:31.516213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:31.517702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:31.517702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:31.517702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:31.517702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:31.517702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:31.517702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:31.517702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:31.517702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:31.517702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:31.517702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:31.517702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:31.517702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:31.517702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:31.517702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:31.517702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:31.517702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:31.517702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:31.519154 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:31.519154 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:31.519154 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:04:31.519154 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 623/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:31.527252 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec9d43-d482-479e-85b4-ea4fb8b0805a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:31.677348 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec9d43-d482-479e-85b4-ea4fb8b0805a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:31.739571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dde85e-81d1-4e12-8b3a-798c0db6444f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:31.740479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dde85e-81d1-4e12-8b3a-798c0db6444f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:31.740963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dde85e-81d1-4e12-8b3a-798c0db6444f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:31.741430 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dde85e-81d1-4e12-8b3a-798c0db6444f None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:31.742692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dde85e-81d1-4e12-8b3a-798c0db6444f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:31.743224 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dde85e-81d1-4e12-8b3a-798c0db6444f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:31.745671 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dde85e-81d1-4e12-8b3a-798c0db6444f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:31.767472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 629/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:31.775052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25df381-7459-403f-9515-d7641781f262 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:31.919272 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25df381-7459-403f-9515-d7641781f262 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:31.965924 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:31.978195 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bc3d0-1f37-48bd-ab54-e362bffaa8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:31.979023 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bc3d0-1f37-48bd-ab54-e362bffaa8e1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:31.979364 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bc3d0-1f37-48bd-ab54-e362bffaa8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:31.979676 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bc3d0-1f37-48bd-ab54-e362bffaa8e1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:31.990781 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bc3d0-1f37-48bd-ab54-e362bffaa8e1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:31.993333 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bc3d0-1f37-48bd-ab54-e362bffaa8e1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:32.009493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-174bc3d0-1f37-48bd-ab54-e362bffaa8e1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:04:32.009493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:32.009493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:32.009493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:32.009493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:32.009493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:32.009493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:32.009493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:32.009493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:32.009493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:32.009493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:32.009493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:32.009493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:04:32.009493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:04:32.009493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:04:32.009493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:04:32.010963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:32.010963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:32.010963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:04:32.010963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:32.010963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:04:32.010963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:32.010963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:32.010963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:32.010963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:32.010963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:32.010963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:32.010963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:32.010963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:32.010963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:32.010963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:32.010963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:32.010963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:32.010963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:32.012622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:32.012622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:32.012622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:04:32.012622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:04:32.012622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:04:32.012622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:04:32.012622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:04:32.012622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:04:32.012622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:04:32.012622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:04:32.012622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 624/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:32.017779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dde85e-81d1-4e12-8b3a-798c0db6444f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:32.113046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:32.183942 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dde85e-81d1-4e12-8b3a-798c0db6444f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:32.231252 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d875c-b3aa-4f43-a3cc-a5aa3ec7218f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:32.232446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d875c-b3aa-4f43-a3cc-a5aa3ec7218f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:32.232724 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d875c-b3aa-4f43-a3cc-a5aa3ec7218f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:32.233125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d875c-b3aa-4f43-a3cc-a5aa3ec7218f None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:32.234281 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d875c-b3aa-4f43-a3cc-a5aa3ec7218f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:32.234792 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d875c-b3aa-4f43-a3cc-a5aa3ec7218f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:32.237123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d875c-b3aa-4f43-a3cc-a5aa3ec7218f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:32.264366 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 630/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Sep 6 16:04:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:32.271279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bc3d0-1f37-48bd-ab54-e362bffaa8e1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:32.418780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bc3d0-1f37-48bd-ab54-e362bffaa8e1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:32.460758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6d48a-b94c-43c6-bb08-584174e22202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:32.461418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6d48a-b94c-43c6-bb08-584174e22202 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:32.461694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6d48a-b94c-43c6-bb08-584174e22202 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:32.461960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6d48a-b94c-43c6-bb08-584174e22202 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:32.535680 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:32.552349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6d48a-b94c-43c6-bb08-584174e22202 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:32.554123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6d48a-b94c-43c6-bb08-584174e22202 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:32.680697 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 625/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:04:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:04:32.687897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d875c-b3aa-4f43-a3cc-a5aa3ec7218f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:32.804691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d875c-b3aa-4f43-a3cc-a5aa3ec7218f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:32.821948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:32.848517 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb88ae1-67c0-4cfa-a837-b2185186d03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:32.849156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb88ae1-67c0-4cfa-a837-b2185186d03d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:32.849443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb88ae1-67c0-4cfa-a837-b2185186d03d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:32.849692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb88ae1-67c0-4cfa-a837-b2185186d03d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:32.861371 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb88ae1-67c0-4cfa-a837-b2185186d03d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:32.863260 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb88ae1-67c0-4cfa-a837-b2185186d03d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:32.886502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-acb88ae1-67c0-4cfa-a837-b2185186d03d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:32.886502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:32.886502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:32.886502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:32.886502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:32.886502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:32.886502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:32.886502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:32.886502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:32.886502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:32.886502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:32.886502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:32.886502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:32.886502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:32.886502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:32.886502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:32.888602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:32.888602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:32.888602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:32.888602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:32.888602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:32.888602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:32.888602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:32.888602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:32.888602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:32.888602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:32.888602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:32.888602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:32.888602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:32.888602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:32.888602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:32.888602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:32.888602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:32.890145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:32.890145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:32.890145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:32.890145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 631/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:32.897501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6d48a-b94c-43c6-bb08-584174e22202 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:33.022105 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:33.037316 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6d48a-b94c-43c6-bb08-584174e22202 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:33.078968 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7464e3-359e-4334-8e7b-a3df815f6b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:33.079694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7464e3-359e-4334-8e7b-a3df815f6b59 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:33.080138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7464e3-359e-4334-8e7b-a3df815f6b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:33.080382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7464e3-359e-4334-8e7b-a3df815f6b59 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:33.081635 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7464e3-359e-4334-8e7b-a3df815f6b59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:33.081947 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7464e3-359e-4334-8e7b-a3df815f6b59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:33.083390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7464e3-359e-4334-8e7b-a3df815f6b59 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:33.100746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 626/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:33.106929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb88ae1-67c0-4cfa-a837-b2185186d03d None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:33.228706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb88ae1-67c0-4cfa-a837-b2185186d03d None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:33.266606 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8f836-cc3e-44f0-921d-b40d67ce26f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:33.267278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8f836-cc3e-44f0-921d-b40d67ce26f6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:33.267659 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8f836-cc3e-44f0-921d-b40d67ce26f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:33.268065 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8f836-cc3e-44f0-921d-b40d67ce26f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:33.277452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8f836-cc3e-44f0-921d-b40d67ce26f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:33.278940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8f836-cc3e-44f0-921d-b40d67ce26f6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:33.293935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-cae8f836-cc3e-44f0-921d-b40d67ce26f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:33.293935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:33.293935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:33.293935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:33.293935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:33.293935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:33.293935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:33.293935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:33.293935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:33.293935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:33.293935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:33.293935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:33.293935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:04:33.293935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:04:33.293935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:04:33.295009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:04:33.295009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:33.295009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:33.295009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:33.295009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:33.295009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:33.295009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:33.295009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:33.295009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:33.295009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:33.295009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:33.295009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:33.295009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:33.295009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:33.295009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:33.295009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:33.295009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:33.295936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:04:33.295936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:04:33.295936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:04:33.295936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:04:33.295936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:04:33.295936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:04:33.295936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:33.295936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:33.295936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 632/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:04:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:33.304091 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7464e3-359e-4334-8e7b-a3df815f6b59 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:33.420398 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7464e3-359e-4334-8e7b-a3df815f6b59 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:33.456077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed888d4d-948a-4f63-9b1e-ce52b8c01731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:33.457031 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed888d4d-948a-4f63-9b1e-ce52b8c01731 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:33.457317 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed888d4d-948a-4f63-9b1e-ce52b8c01731 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:33.457583 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed888d4d-948a-4f63-9b1e-ce52b8c01731 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:33.458274 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed888d4d-948a-4f63-9b1e-ce52b8c01731 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:33.458574 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed888d4d-948a-4f63-9b1e-ce52b8c01731 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:33.460045 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed888d4d-948a-4f63-9b1e-ce52b8c01731 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:33.472852 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 627/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:04:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:33.479157 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8f836-cc3e-44f0-921d-b40d67ce26f6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:33.516267 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:33.595039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8f836-cc3e-44f0-921d-b40d67ce26f6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:33.633424 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ac0d6-13fb-453e-962e-797b65e06863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:33.634030 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ac0d6-13fb-453e-962e-797b65e06863 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:33.634308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ac0d6-13fb-453e-962e-797b65e06863 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:33.634579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ac0d6-13fb-453e-962e-797b65e06863 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:33.635377 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ac0d6-13fb-453e-962e-797b65e06863 None admin] RBAC: Adding query filter params (role.id=ba05677650e046a48d5f19440d48fbf2, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=034d75ab747e4c0cab040e47c6a6236e) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:33.635737 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ac0d6-13fb-453e-962e-797b65e06863 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:33.637245 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ac0d6-13fb-453e-962e-797b65e06863 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:33.643165 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 633/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:04:33 2022] GET /identity/v3/role_assignments?user.id=034d75ab747e4c0cab040e47c6a6236e&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:35.703932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27916785-7d2f-4d25-8d74-4656f3f3a6db None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:35.704456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27916785-7d2f-4d25-8d74-4656f3f3a6db None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:35.704456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27916785-7d2f-4d25-8d74-4656f3f3a6db None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:35.706325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 628/1261] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:04:35.711779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fea725-0c83-4146-a840-4b34c280dab4 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:35.712235 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fea725-0c83-4146-a840-4b34c280dab4 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:35.712327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fea725-0c83-4146-a840-4b34c280dab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:35.731446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-38fea725-0c83-4146-a840-4b34c280dab4 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:35.732634 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-38fea725-0c83-4146-a840-4b34c280dab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:35.800471 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 634/1262] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:36.081264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27916785-7d2f-4d25-8d74-4656f3f3a6db None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:36.092437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:36.184207 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27916785-7d2f-4d25-8d74-4656f3f3a6db None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:36.217400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1ed2f-d3e9-43a3-8fce-d7f19f3dd5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:36.218029 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1ed2f-d3e9-43a3-8fce-d7f19f3dd5f9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:36.218303 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1ed2f-d3e9-43a3-8fce-d7f19f3dd5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:36.218575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1ed2f-d3e9-43a3-8fce-d7f19f3dd5f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:36.229448 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1ed2f-d3e9-43a3-8fce-d7f19f3dd5f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:04:36.234300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-f7d1ed2f-d3e9-43a3-8fce-d7f19f3dd5f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:04:36.234300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:36.234300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:36.234300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:36.234300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:36.234300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:36.234300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:36.234300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:36.234300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:36.234300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:36.234300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:36.234300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:36.234300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:04:36.234300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:04:36.234300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:04:36.234300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:04:36.235443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:04:36.235443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:04:36.235443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:04:36.235443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:04:36.235443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:36.235443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:36.235443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:36.235443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:36.235443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:36.235443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:36.235443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:36.235443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:36.235443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:36.235443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:36.235443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:36.235443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:36.236325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:36.236325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:36.236325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:36.236325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:36.236325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:04:36.236325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:04:36.236325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:04:36.236325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:04:36.236325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:04:36.236325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:04:36.236325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:04:36.236325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:04:36.236325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 629/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:04:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:36.241629 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fea725-0c83-4146-a840-4b34c280dab4 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:36.260430 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:36.352952 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fea725-0c83-4146-a840-4b34c280dab4 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:36.387374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fbe11-a388-431b-b49c-e7d063ac145a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:36.387970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fbe11-a388-431b-b49c-e7d063ac145a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:36.388248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fbe11-a388-431b-b49c-e7d063ac145a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:36.388514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fbe11-a388-431b-b49c-e7d063ac145a None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:36.389162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fbe11-a388-431b-b49c-e7d063ac145a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:36.389463 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fbe11-a388-431b-b49c-e7d063ac145a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:36.390838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fbe11-a388-431b-b49c-e7d063ac145a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:36.399410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 635/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:04:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:36.405150 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1ed2f-d3e9-43a3-8fce-d7f19f3dd5f9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:36.472299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:36.506142 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1ed2f-d3e9-43a3-8fce-d7f19f3dd5f9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:36.539342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc9a53-be47-4b14-bf4f-b0e27e2d9ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:36.539971 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc9a53-be47-4b14-bf4f-b0e27e2d9ebb None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:36.540249 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc9a53-be47-4b14-bf4f-b0e27e2d9ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:36.540520 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc9a53-be47-4b14-bf4f-b0e27e2d9ebb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:36.550896 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc9a53-be47-4b14-bf4f-b0e27e2d9ebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:36.552332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc9a53-be47-4b14-bf4f-b0e27e2d9ebb None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:36.566752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-f2fc9a53-be47-4b14-bf4f-b0e27e2d9ebb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:36.566752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:36.566752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:36.566752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:36.566752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:36.566752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:36.566752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:36.566752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:36.566752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:36.566752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:36.566752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:36.566752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:36.566752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:36.566752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:36.566752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:36.566752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:36.567986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:36.567986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:36.567986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:36.567986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:36.567986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:36.567986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:36.567986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:36.567986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:36.567986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:36.567986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:36.567986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:36.567986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:36.567986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:36.567986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:36.567986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:36.567986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:36.567986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:36.568960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:36.568960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:36.568960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:04:36.568960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 630/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:36.574368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fbe11-a388-431b-b49c-e7d063ac145a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:36.654678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:36.674109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fbe11-a388-431b-b49c-e7d063ac145a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:36.707848 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6204df-3a8e-42f9-8c30-9d5453ede6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:36.708437 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6204df-3a8e-42f9-8c30-9d5453ede6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:36.708715 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6204df-3a8e-42f9-8c30-9d5453ede6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:36.708985 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6204df-3a8e-42f9-8c30-9d5453ede6e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:36.709614 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6204df-3a8e-42f9-8c30-9d5453ede6e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:36.709914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6204df-3a8e-42f9-8c30-9d5453ede6e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:36.711291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6204df-3a8e-42f9-8c30-9d5453ede6e3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:36.723422 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 636/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:36.729112 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc9a53-be47-4b14-bf4f-b0e27e2d9ebb None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:36.826576 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc9a53-be47-4b14-bf4f-b0e27e2d9ebb None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:36.846067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:36.861037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef5112-1100-4831-a53f-61e6f1c77193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:36.861683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef5112-1100-4831-a53f-61e6f1c77193 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:36.861962 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef5112-1100-4831-a53f-61e6f1c77193 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:36.862685 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef5112-1100-4831-a53f-61e6f1c77193 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:36.869822 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef5112-1100-4831-a53f-61e6f1c77193 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:36.871359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef5112-1100-4831-a53f-61e6f1c77193 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:36.882854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-3fef5112-1100-4831-a53f-61e6f1c77193 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:04:36.882854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:36.882854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:36.882854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:36.882854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:36.882854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:36.882854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:36.882854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:36.882854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:36.882854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:36.882854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:36.882854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:36.882854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:04:36.882854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:04:36.882854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:04:36.882854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:04:36.884457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:36.884457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:36.884457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:04:36.884457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:36.884457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:04:36.884457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:36.884457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:36.884457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:36.884457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:36.884457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:36.884457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:36.884457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:36.884457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:36.884457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:36.884457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:36.884457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:36.884457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:36.884457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:36.885461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:36.885461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:36.885461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:04:36.885461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:04:36.885461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:04:36.885461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:04:36.885461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:04:36.885461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:04:36.885461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:04:36.885461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:04:36.885461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 631/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:36.889835 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6204df-3a8e-42f9-8c30-9d5453ede6e3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:36.995163 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6204df-3a8e-42f9-8c30-9d5453ede6e3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:37.028346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72d4c5-d3cf-4ad4-a60e-1f7e09f10c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:37.028938 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72d4c5-d3cf-4ad4-a60e-1f7e09f10c4d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:37.029215 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72d4c5-d3cf-4ad4-a60e-1f7e09f10c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:37.029483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72d4c5-d3cf-4ad4-a60e-1f7e09f10c4d None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:37.030192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72d4c5-d3cf-4ad4-a60e-1f7e09f10c4d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:37.030492 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72d4c5-d3cf-4ad4-a60e-1f7e09f10c4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:37.031925 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72d4c5-d3cf-4ad4-a60e-1f7e09f10c4d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:37.041991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:37.051558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 637/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Sep 6 16:04:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:37.057510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef5112-1100-4831-a53f-61e6f1c77193 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:37.155279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef5112-1100-4831-a53f-61e6f1c77193 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:37.190055 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb03409-bb0b-4d23-bfed-810ff1ed9999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:37.191476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb03409-bb0b-4d23-bfed-810ff1ed9999 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:37.191749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb03409-bb0b-4d23-bfed-810ff1ed9999 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:37.192020 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb03409-bb0b-4d23-bfed-810ff1ed9999 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:37.202407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb03409-bb0b-4d23-bfed-810ff1ed9999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:37.203894 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb03409-bb0b-4d23-bfed-810ff1ed9999 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:37.218001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-5cb03409-bb0b-4d23-bfed-810ff1ed9999 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:37.218001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:37.218001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:37.218001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:37.218001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:37.218001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:37.218001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:37.218001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:37.218001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:37.218001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:37.218001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:37.218001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:37.218001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:37.218001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:37.218001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:37.218001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:37.219149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:37.219149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:37.219149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:37.219149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:37.219149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:37.219149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:37.219149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:37.219149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:37.219149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:37.219149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:37.219149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:37.219149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:37.219149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:37.219149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:37.219149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:37.219149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:37.219149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:37.220225 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:37.220225 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:37.220225 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:04:37.220225 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 632/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:37.225330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72d4c5-d3cf-4ad4-a60e-1f7e09f10c4d None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:37.341296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72d4c5-d3cf-4ad4-a60e-1f7e09f10c4d None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:37.375322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7406513f-97ee-44b1-8098-f908552903f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:37.375925 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406513f-97ee-44b1-8098-f908552903f1 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:37.376206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406513f-97ee-44b1-8098-f908552903f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:37.376477 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406513f-97ee-44b1-8098-f908552903f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:37.377113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7406513f-97ee-44b1-8098-f908552903f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:37.377414 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7406513f-97ee-44b1-8098-f908552903f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:37.379423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7406513f-97ee-44b1-8098-f908552903f1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:37.390629 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 638/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:37.396455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb03409-bb0b-4d23-bfed-810ff1ed9999 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:37.428142 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:37.497494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb03409-bb0b-4d23-bfed-810ff1ed9999 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:37.531254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efd108-64ff-4bc4-a009-e630b5c186ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:37.531880 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efd108-64ff-4bc4-a009-e630b5c186ed None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:37.532157 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efd108-64ff-4bc4-a009-e630b5c186ed None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:37.532427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efd108-64ff-4bc4-a009-e630b5c186ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:37.539794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22efd108-64ff-4bc4-a009-e630b5c186ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:37.541226 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22efd108-64ff-4bc4-a009-e630b5c186ed None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:37.553459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-22efd108-64ff-4bc4-a009-e630b5c186ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:37.553459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:37.553459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:37.553459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:37.553459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:37.553459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:37.553459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:37.553459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:37.553459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:37.553459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:37.553459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:37.553459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:37.553459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:04:37.553459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:04:37.553459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:04:37.554841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:04:37.554841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:37.554841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:37.554841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:37.554841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:37.554841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:37.554841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:37.554841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:37.554841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:37.554841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:37.554841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:37.554841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:37.554841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:37.554841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:37.554841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:37.554841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:37.554841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:37.555995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:04:37.555995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:04:37.555995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:04:37.555995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:04:37.555995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:04:37.555995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:04:37.555995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:37.555995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:04:37.555995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 633/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:04:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:37.560694 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7406513f-97ee-44b1-8098-f908552903f1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:37.621188 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:37.663566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7406513f-97ee-44b1-8098-f908552903f1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:37.697519 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369abd8d-3fb0-4846-89bc-3f29f381c8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:37.698124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369abd8d-3fb0-4846-89bc-3f29f381c8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:37.698402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369abd8d-3fb0-4846-89bc-3f29f381c8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:37.698693 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369abd8d-3fb0-4846-89bc-3f29f381c8f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:37.699387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369abd8d-3fb0-4846-89bc-3f29f381c8f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:37.699695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369abd8d-3fb0-4846-89bc-3f29f381c8f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:37.701116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369abd8d-3fb0-4846-89bc-3f29f381c8f6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:37.713175 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 639/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:04:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:37.719355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efd108-64ff-4bc4-a009-e630b5c186ed None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:37.814010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:37.823696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efd108-64ff-4bc4-a009-e630b5c186ed None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:37.857090 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9702aa0-6996-4fce-b215-5a087af959a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:37.857663 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9702aa0-6996-4fce-b215-5a087af959a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:37.857946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9702aa0-6996-4fce-b215-5a087af959a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:37.858221 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9702aa0-6996-4fce-b215-5a087af959a7 None admin] PATH_INFO: `/v3/projects/0138d62d5879495499a17e20310d4f61/users/034d75ab747e4c0cab040e47c6a6236e/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:37.860120 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9702aa0-6996-4fce-b215-5a087af959a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=0138d62d5879495499a17e20310d4f61, user_id=034d75ab747e4c0cab040e47c6a6236e, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:37.861584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9702aa0-6996-4fce-b215-5a087af959a7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:37.876958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 634/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 6 16:04:37 2022] PUT /identity/v3/projects/0138d62d5879495499a17e20310d4f61/users/034d75ab747e4c0cab040e47c6a6236e/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:04:39.818883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0611c-9b29-4da0-b978-1dc47dd46525 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:39.819444 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0611c-9b29-4da0-b978-1dc47dd46525 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:39.819444 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0611c-9b29-4da0-b978-1dc47dd46525 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:39.820782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 640/1274] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:39.825891 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e506ff-e299-418e-a8da-c23830562801 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:39.826316 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e506ff-e299-418e-a8da-c23830562801 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:39.826433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e506ff-e299-418e-a8da-c23830562801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:39.845667 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-47e506ff-e299-418e-a8da-c23830562801 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:39.847519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-47e506ff-e299-418e-a8da-c23830562801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:39.917308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 635/1275] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:40.201877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0611c-9b29-4da0-b978-1dc47dd46525 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:40.319676 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0611c-9b29-4da0-b978-1dc47dd46525 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:40.343474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:40.367683 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c47c6-5fb8-4b05-bf52-047a5b4e69c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:40.368329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c47c6-5fb8-4b05-bf52-047a5b4e69c9 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:40.368609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c47c6-5fb8-4b05-bf52-047a5b4e69c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:40.368879 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c47c6-5fb8-4b05-bf52-047a5b4e69c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:40.380965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c47c6-5fb8-4b05-bf52-047a5b4e69c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 16:04:40.385790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-ed6c47c6-5fb8-4b05-bf52-047a5b4e69c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:04:40.385790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:40.385790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:40.385790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:40.385790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:40.385790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:40.385790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:40.385790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:40.385790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:40.385790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:40.385790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:40.385790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:40.385790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 16:04:40.385790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 16:04:40.385790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 16:04:40.385790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 16:04:40.386928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 16:04:40.386928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 16:04:40.386928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 16:04:40.386928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 16:04:40.386928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:40.386928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:40.386928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:40.386928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:40.386928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:40.386928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:40.386928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:40.386928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:40.386928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:40.386928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:40.386928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:40.386928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:40.388119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:40.388119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:40.388119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:40.388119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:40.388119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 16:04:40.388119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 16:04:40.388119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 16:04:40.388119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 16:04:40.388119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 16:04:40.388119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 16:04:40.388119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 16:04:40.388119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:40.388119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 641/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 6 16:04:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:40.392997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e506ff-e299-418e-a8da-c23830562801 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:40.475169 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:40.493237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e506ff-e299-418e-a8da-c23830562801 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:40.526454 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323aa39-5c9a-4fde-ae60-36c2943863ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:40.527075 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323aa39-5c9a-4fde-ae60-36c2943863ad None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:40.527350 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323aa39-5c9a-4fde-ae60-36c2943863ad None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:40.527616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323aa39-5c9a-4fde-ae60-36c2943863ad None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:40.528264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7323aa39-5c9a-4fde-ae60-36c2943863ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:40.528561 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7323aa39-5c9a-4fde-ae60-36c2943863ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:40.529920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7323aa39-5c9a-4fde-ae60-36c2943863ad None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:40.538863 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 636/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 6 16:04:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:40.544584 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c47c6-5fb8-4b05-bf52-047a5b4e69c9 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:40.646899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c47c6-5fb8-4b05-bf52-047a5b4e69c9 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:40.680757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a208d-a64d-4029-ad21-19ef18d012af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:40.681368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a208d-a64d-4029-ad21-19ef18d012af None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:40.681640 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a208d-a64d-4029-ad21-19ef18d012af None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:40.681904 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a208d-a64d-4029-ad21-19ef18d012af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:40.692273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a208d-a64d-4029-ad21-19ef18d012af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:40.693704 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a208d-a64d-4029-ad21-19ef18d012af None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:40.708739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-665a208d-a64d-4029-ad21-19ef18d012af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:40.708739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:40.708739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:40.708739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:40.708739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:40.708739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:40.708739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:40.708739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:40.708739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:40.708739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:40.708739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:40.708739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:40.708739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:40.708739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:40.708739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:40.708739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:40.709877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:40.709877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:40.709877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:40.709877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:40.709877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:40.709877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:40.709877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:40.709877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:40.709877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:40.709877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:40.709877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:40.709877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:40.709877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:40.709877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:40.709877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:40.709877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:40.709877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:40.711222 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:40.711222 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:40.711222 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:40.711222 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 642/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:40.715858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323aa39-5c9a-4fde-ae60-36c2943863ad None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:40.814443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323aa39-5c9a-4fde-ae60-36c2943863ad None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:40.847758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4b3eb-d7e5-411e-bc90-a86579c90061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:40.848380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4b3eb-d7e5-411e-bc90-a86579c90061 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:40.848629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4b3eb-d7e5-411e-bc90-a86579c90061 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:40.848902 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4b3eb-d7e5-411e-bc90-a86579c90061 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:40.849536 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4b3eb-d7e5-411e-bc90-a86579c90061 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:40.849839 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4b3eb-d7e5-411e-bc90-a86579c90061 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:40.851243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4b3eb-d7e5-411e-bc90-a86579c90061 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:40.862904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 637/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:40.868767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a208d-a64d-4029-ad21-19ef18d012af None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:40.900428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:40.972410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a208d-a64d-4029-ad21-19ef18d012af None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:41.005560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f039dd-c46a-44dc-9516-06da81d0b962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:41.006176 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f039dd-c46a-44dc-9516-06da81d0b962 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:41.006455 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f039dd-c46a-44dc-9516-06da81d0b962 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:41.006753 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f039dd-c46a-44dc-9516-06da81d0b962 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:41.013939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f039dd-c46a-44dc-9516-06da81d0b962 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:41.015397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f039dd-c46a-44dc-9516-06da81d0b962 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:41.026566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-42f039dd-c46a-44dc-9516-06da81d0b962 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:04:41.026566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:41.026566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:41.026566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:41.026566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:41.026566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:41.026566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:41.026566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:41.026566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:41.026566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:41.026566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:41.026566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:41.026566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 16:04:41.026566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 16:04:41.026566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 16:04:41.026566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 16:04:41.027691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:41.027691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:41.027691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 16:04:41.027691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:41.027691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 16:04:41.027691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 16:04:41.027691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:41.027691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:41.027691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:41.027691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:41.027691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:41.027691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:41.027691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:41.027691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:41.027691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:41.027691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:41.027691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:41.027691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:41.028635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:41.028635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:41.028635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 16:04:41.028635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 16:04:41.028635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 16:04:41.028635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 16:04:41.028635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 16:04:41.028635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 16:04:41.028635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 16:04:41.028635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:41.028635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 643/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:41.033554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4b3eb-d7e5-411e-bc90-a86579c90061 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:41.037465 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:41.133429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4b3eb-d7e5-411e-bc90-a86579c90061 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:41.166411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e9535-12ee-4c1e-b003-4803eca648c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:41.167038 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e9535-12ee-4c1e-b003-4803eca648c2 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:41.167320 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e9535-12ee-4c1e-b003-4803eca648c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:41.167593 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e9535-12ee-4c1e-b003-4803eca648c2 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:41.168313 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e9535-12ee-4c1e-b003-4803eca648c2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:41.168620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e9535-12ee-4c1e-b003-4803eca648c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:41.170047 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e9535-12ee-4c1e-b003-4803eca648c2 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:41.187662 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 638/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Sep 6 16:04:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:41.194094 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f039dd-c46a-44dc-9516-06da81d0b962 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:41.289144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:41.301990 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f039dd-c46a-44dc-9516-06da81d0b962 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:41.344560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779c220-e2c1-42de-89b7-6326cf24ca96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:41.345155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779c220-e2c1-42de-89b7-6326cf24ca96 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:41.345434 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779c220-e2c1-42de-89b7-6326cf24ca96 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:41.345710 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779c220-e2c1-42de-89b7-6326cf24ca96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:41.413573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779c220-e2c1-42de-89b7-6326cf24ca96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:41.414984 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779c220-e2c1-42de-89b7-6326cf24ca96 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:41.502263 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:41.513027 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 644/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 6 16:04:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 16:04:41.519045 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e9535-12ee-4c1e-b003-4803eca648c2 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:41.580570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:41.620841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e9535-12ee-4c1e-b003-4803eca648c2 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:41.654511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17265da5-34e2-4db4-81dd-ee27ba59217b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:41.655346 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17265da5-34e2-4db4-81dd-ee27ba59217b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:41.655627 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17265da5-34e2-4db4-81dd-ee27ba59217b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:41.655899 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17265da5-34e2-4db4-81dd-ee27ba59217b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:41.666854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17265da5-34e2-4db4-81dd-ee27ba59217b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:41.668243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17265da5-34e2-4db4-81dd-ee27ba59217b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:41.683032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-17265da5-34e2-4db4-81dd-ee27ba59217b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:41.683032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:41.683032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:41.683032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:41.683032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:41.683032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:41.683032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:41.683032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:41.683032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:41.683032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:41.683032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:41.683032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:41.683032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 16:04:41.683032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 16:04:41.683032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 16:04:41.683032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 16:04:41.684132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:41.684132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:41.684132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:41.684132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:41.684132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:41.684132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:41.684132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:41.684132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:41.684132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:41.684132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:41.684132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:41.684132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:41.684132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:41.684132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:41.684132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:41.684132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:41.684132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 16:04:41.685199 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 16:04:41.685199 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 16:04:41.685199 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:04:41.685199 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 639/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:04:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:41.690099 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779c220-e2c1-42de-89b7-6326cf24ca96 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:41.789428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779c220-e2c1-42de-89b7-6326cf24ca96 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:41.822820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aa108-5d1f-4c9e-9c14-179eeddee935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:41.823420 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aa108-5d1f-4c9e-9c14-179eeddee935 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:41.823699 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aa108-5d1f-4c9e-9c14-179eeddee935 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:41.823971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aa108-5d1f-4c9e-9c14-179eeddee935 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:41.824607 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0aa108-5d1f-4c9e-9c14-179eeddee935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:41.824909 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0aa108-5d1f-4c9e-9c14-179eeddee935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:41.826254 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0aa108-5d1f-4c9e-9c14-179eeddee935 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:41.838013 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 645/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 6 16:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:41.843696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17265da5-34e2-4db4-81dd-ee27ba59217b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:41.947709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17265da5-34e2-4db4-81dd-ee27ba59217b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:41.961843 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:41.982950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26e697-062b-42b8-9b1b-d0c6677f12b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:41.983582 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26e697-062b-42b8-9b1b-d0c6677f12b7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:41.983862 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26e697-062b-42b8-9b1b-d0c6677f12b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:41.984133 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26e697-062b-42b8-9b1b-d0c6677f12b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:41.990710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26e697-062b-42b8-9b1b-d0c6677f12b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:41.992153 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26e697-062b-42b8-9b1b-d0c6677f12b7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:42.003230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-3c26e697-062b-42b8-9b1b-d0c6677f12b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:42.003230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:42.003230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:42.003230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:42.003230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:42.003230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:42.003230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:42.003230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:42.003230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:42.003230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:42.003230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:42.003230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:42.003230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 16:04:42.003230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 16:04:42.003230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 16:04:42.004250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 16:04:42.004250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:42.004250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:42.004250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:42.004250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:42.004250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:42.004250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:42.004250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:42.004250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:42.004250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:42.004250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:42.004250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:42.004250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:42.004250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:42.004250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:42.004250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:42.004250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:42.005401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 16:04:42.005401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 16:04:42.005401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 16:04:42.005401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 16:04:42.005401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 16:04:42.005401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 16:04:42.005401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 16:04:42.005401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Sep 06 16:04:42.005401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 640/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 6 16:04:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:42.010420 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aa108-5d1f-4c9e-9c14-179eeddee935 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:42.061119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:42.112957 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aa108-5d1f-4c9e-9c14-179eeddee935 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:42.146800 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa088a-db94-40b9-96f6-7a60f1fd057c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:42.147438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa088a-db94-40b9-96f6-7a60f1fd057c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:42.147684 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa088a-db94-40b9-96f6-7a60f1fd057c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:42.147958 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa088a-db94-40b9-96f6-7a60f1fd057c None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:42.148640 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa088a-db94-40b9-96f6-7a60f1fd057c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:42.148947 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa088a-db94-40b9-96f6-7a60f1fd057c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:42.150368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa088a-db94-40b9-96f6-7a60f1fd057c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:42.162342 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 646/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 6 16:04:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:42.168759 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26e697-062b-42b8-9b1b-d0c6677f12b7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:42.270814 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26e697-062b-42b8-9b1b-d0c6677f12b7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:42.303690 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09dc3e-ced9-40e2-9016-f5da76007d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:42.304283 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09dc3e-ced9-40e2-9016-f5da76007d22 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:42.304814 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09dc3e-ced9-40e2-9016-f5da76007d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:42.305077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09dc3e-ced9-40e2-9016-f5da76007d22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:42.305776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09dc3e-ced9-40e2-9016-f5da76007d22 None admin] RBAC: Adding query filter params (role.id=ba05677650e046a48d5f19440d48fbf2, scope.project.id=0138d62d5879495499a17e20310d4f61, user.id=034d75ab747e4c0cab040e47c6a6236e) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:42.306076 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09dc3e-ced9-40e2-9016-f5da76007d22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:42.307511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09dc3e-ced9-40e2-9016-f5da76007d22 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:42.313148 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 641/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 6 16:04:42 2022] GET /identity/v3/role_assignments?user.id=034d75ab747e4c0cab040e47c6a6236e&scope.project.id=0138d62d5879495499a17e20310d4f61&role.id=ba05677650e046a48d5f19440d48fbf2 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:44.261456 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a4189-e436-4dcf-b156-560db9003a27 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:44.261987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a4189-e436-4dcf-b156-560db9003a27 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:44.261987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a4189-e436-4dcf-b156-560db9003a27 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:44.263386 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 647/1288] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:44.269068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b06d0-7dfd-47ef-bd52-8b2bc93f71ab None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:44.269479 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b06d0-7dfd-47ef-bd52-8b2bc93f71ab None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:44.269611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b06d0-7dfd-47ef-bd52-8b2bc93f71ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:44.288406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-7a6b06d0-7dfd-47ef-bd52-8b2bc93f71ab None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:44.289602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7a6b06d0-7dfd-47ef-bd52-8b2bc93f71ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:44.319922 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:44.358092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 642/1289] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:44.625699 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a4189-e436-4dcf-b156-560db9003a27 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:44.726920 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a4189-e436-4dcf-b156-560db9003a27 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:44.737700 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:44.761098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ed929-2740-4b4b-b845-ba05982f5942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:44.761728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ed929-2740-4b4b-b845-ba05982f5942 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:44.762008 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ed929-2740-4b4b-b845-ba05982f5942 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:44.762281 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ed929-2740-4b4b-b845-ba05982f5942 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:44.762893 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ed929-2740-4b4b-b845-ba05982f5942 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:44.764263 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ed929-2740-4b4b-b845-ba05982f5942 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:44.773583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-e92ed929-2740-4b4b-b845-ba05982f5942 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 16:04:44.773583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:44.773583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:04:44.773583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:04:44.773583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:04:44.773583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:04:44.773583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:04:44.773583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:44.773583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 16:04:44.773583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:44.773583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:44.773583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:44.773583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:44.773583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:44.773583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:44.773583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:44.773583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:44.774910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:44.774910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:44.774910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:44.774910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:44.774910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:44.774910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:44.774910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:04:44.774910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:04:44.774910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:04:44.774910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:04:44.774910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:44.774910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:44.774910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:44.774910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:44.774910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:44.774910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:44.776045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:44.776045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:44.776045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:44.776045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:44.776045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:44.776045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:44.776045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:44.776045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:44.776045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:44.776045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:44.776045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:04:44.776045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:44.776045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 16:04:44.776045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:44.776045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 648/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 6 16:04:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:44.780700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b06d0-7dfd-47ef-bd52-8b2bc93f71ab None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:44.878823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b06d0-7dfd-47ef-bd52-8b2bc93f71ab None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:44.913771 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e58aa-0fd8-4f90-8c0a-6647555b8ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:44.914461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e58aa-0fd8-4f90-8c0a-6647555b8ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:44.914771 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e58aa-0fd8-4f90-8c0a-6647555b8ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:44.915057 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e58aa-0fd8-4f90-8c0a-6647555b8ab1 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:44.915698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006e58aa-0fd8-4f90-8c0a-6647555b8ab1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:44.916006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006e58aa-0fd8-4f90-8c0a-6647555b8ab1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:44.917402 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006e58aa-0fd8-4f90-8c0a-6647555b8ab1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:44.922579 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 643/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Tue Sep 6 16:04:44 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:44.929179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ed929-2740-4b4b-b845-ba05982f5942 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:45.030333 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ed929-2740-4b4b-b845-ba05982f5942 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:45.063918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927acc8d-7557-4889-a6e7-f714a6a68b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:45.064515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927acc8d-7557-4889-a6e7-f714a6a68b55 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:45.064795 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927acc8d-7557-4889-a6e7-f714a6a68b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:45.065569 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927acc8d-7557-4889-a6e7-f714a6a68b55 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:45.066201 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927acc8d-7557-4889-a6e7-f714a6a68b55 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:45.066503 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927acc8d-7557-4889-a6e7-f714a6a68b55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:45.067892 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927acc8d-7557-4889-a6e7-f714a6a68b55 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:45.072200 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 649/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Tue Sep 6 16:04:44 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:47.011833 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca5cb5-dd14-4c18-a371-2d75802f0fc6 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:47.012356 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca5cb5-dd14-4c18-a371-2d75802f0fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:47.012356 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca5cb5-dd14-4c18-a371-2d75802f0fc6 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:47.013628 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 644/1293] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:47.019148 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759df9d-39db-4fb8-976d-4b11e60c734b None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:47.019567 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759df9d-39db-4fb8-976d-4b11e60c734b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:47.019680 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759df9d-39db-4fb8-976d-4b11e60c734b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:47.037937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e759df9d-39db-4fb8-976d-4b11e60c734b None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:47.039166 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e759df9d-39db-4fb8-976d-4b11e60c734b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:47.069549 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:47.106425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 650/1294] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:47.419998 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca5cb5-dd14-4c18-a371-2d75802f0fc6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:47.451709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:47.526732 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca5cb5-dd14-4c18-a371-2d75802f0fc6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:47.559002 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e20bb2-d654-4863-b869-ea4d7831e560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:47.559593 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e20bb2-d654-4863-b869-ea4d7831e560 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:47.559866 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e20bb2-d654-4863-b869-ea4d7831e560 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:47.560133 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e20bb2-d654-4863-b869-ea4d7831e560 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:47.560778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e20bb2-d654-4863-b869-ea4d7831e560 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:47.562149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e20bb2-d654-4863-b869-ea4d7831e560 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:47.577764 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 645/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 6 16:04:47 2022] POST /identity/v3/services => generated 246 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:04:49.540107 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65160a-47ec-416e-bf08-4296ceee9394 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:49.540633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65160a-47ec-416e-bf08-4296ceee9394 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:49.540633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65160a-47ec-416e-bf08-4296ceee9394 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:49.542002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 651/1296] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:49.547739 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027f845-2f85-47c3-8987-47088c7a9632 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:49.548265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027f845-2f85-47c3-8987-47088c7a9632 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:49.548265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027f845-2f85-47c3-8987-47088c7a9632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:49.567526 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-2027f845-2f85-47c3-8987-47088c7a9632 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:49.568701 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2027f845-2f85-47c3-8987-47088c7a9632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:49.641183 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 646/1297] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:49 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:49.912621 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65160a-47ec-416e-bf08-4296ceee9394 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:50.026467 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65160a-47ec-416e-bf08-4296ceee9394 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:50.066752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d821f-bf61-453a-b60f-f5c8f449aff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:50.067379 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d821f-bf61-453a-b60f-f5c8f449aff4 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:50.067654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d821f-bf61-453a-b60f-f5c8f449aff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:50.067923 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d821f-bf61-453a-b60f-f5c8f449aff4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:50.068500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d821f-bf61-453a-b60f-f5c8f449aff4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:50.069861 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d821f-bf61-453a-b60f-f5c8f449aff4 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:50.078511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-7e9d821f-bf61-453a-b60f-f5c8f449aff4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 16:04:50.078511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:50.078511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:04:50.078511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:04:50.078511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:04:50.078511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:04:50.078511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:04:50.078511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:50.078511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 16:04:50.078511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:50.078511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:50.078511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:50.078511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:50.078511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:50.078511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:50.078511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:50.078511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:50.079823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:50.079823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:50.079823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:50.079823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:50.079823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:50.079823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:50.079823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:04:50.079823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:04:50.079823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:04:50.079823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:04:50.079823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:50.079823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:50.079823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:50.079823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:50.079823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:50.079823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:50.081533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:50.081533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:50.081533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:50.081533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:50.081533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:50.081533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:50.081533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:50.081533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:50.081533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:50.081533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:50.081533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:04:50.081533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:50.081533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 16:04:50.081533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:50.081533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 652/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 6 16:04:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:50.086499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027f845-2f85-47c3-8987-47088c7a9632 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:50.105022 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:50.187536 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027f845-2f85-47c3-8987-47088c7a9632 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:50.221540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552beae-db30-4171-a7f8-9a338cbc9bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:50.222745 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552beae-db30-4171-a7f8-9a338cbc9bcd None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:50.223029 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552beae-db30-4171-a7f8-9a338cbc9bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:50.223304 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552beae-db30-4171-a7f8-9a338cbc9bcd None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:50.223939 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552beae-db30-4171-a7f8-9a338cbc9bcd None admin] RBAC: Adding query filter params (name=placement) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:50.224245 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552beae-db30-4171-a7f8-9a338cbc9bcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:50.225620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552beae-db30-4171-a7f8-9a338cbc9bcd None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:50.230869 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 647/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Tue Sep 6 16:04:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:50.236990 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d821f-bf61-453a-b60f-f5c8f449aff4 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:50.267607 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:50.350233 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d821f-bf61-453a-b60f-f5c8f449aff4 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:50.389599 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7bbdf-87a6-4e99-8c7f-afb82e2eb3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:50.390197 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7bbdf-87a6-4e99-8c7f-afb82e2eb3a0 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:50.390477 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7bbdf-87a6-4e99-8c7f-afb82e2eb3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:50.390780 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7bbdf-87a6-4e99-8c7f-afb82e2eb3a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:50.391480 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7bbdf-87a6-4e99-8c7f-afb82e2eb3a0 None admin] RBAC: Adding query filter params (interface=public, service_id=4696aee9abaf45a5a7ce914dc8b8a4d1, region_id=RegionOne) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:50.391787 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7bbdf-87a6-4e99-8c7f-afb82e2eb3a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:50.393158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7bbdf-87a6-4e99-8c7f-afb82e2eb3a0 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:50.399617 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 653/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Sep 6 16:04:50 2022] GET /identity/v3/endpoints?service_id=4696aee9abaf45a5a7ce914dc8b8a4d1&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:50.405449 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552beae-db30-4171-a7f8-9a338cbc9bcd None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:50.487330 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:50.506123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552beae-db30-4171-a7f8-9a338cbc9bcd None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:50.539337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22196cc6-2b1c-4270-9189-a0cdfc24194c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:50.539936 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22196cc6-2b1c-4270-9189-a0cdfc24194c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:50.540209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22196cc6-2b1c-4270-9189-a0cdfc24194c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:50.540476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22196cc6-2b1c-4270-9189-a0cdfc24194c None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:50.541067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22196cc6-2b1c-4270-9189-a0cdfc24194c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:50.542416 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22196cc6-2b1c-4270-9189-a0cdfc24194c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:50.548981 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 648/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Sep 6 16:04:50 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:04:52.473303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5bf46-4057-477e-aeda-4efe45c0113e None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:52.473910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5bf46-4057-477e-aeda-4efe45c0113e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:52.473910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5bf46-4057-477e-aeda-4efe45c0113e None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:52.475220 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 654/1302] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:04:52.480572 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbbe86-c3aa-49da-b228-8980b50bf958 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:52.481034 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbbe86-c3aa-49da-b228-8980b50bf958 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:52.481155 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbbe86-c3aa-49da-b228-8980b50bf958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:52.499727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-facbbe86-c3aa-49da-b228-8980b50bf958 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:04:52.500907 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-facbbe86-c3aa-49da-b228-8980b50bf958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:04:52.571003 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 649/1303] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:04:52 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 16:04:52.839479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5bf46-4057-477e-aeda-4efe45c0113e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:52.936794 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:52.944720 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5bf46-4057-477e-aeda-4efe45c0113e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:52.981427 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd4ba8-7a13-4cc4-a417-9e5f670b07c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:52.982052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd4ba8-7a13-4cc4-a417-9e5f670b07c3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:52.982328 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd4ba8-7a13-4cc4-a417-9e5f670b07c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:52.982597 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd4ba8-7a13-4cc4-a417-9e5f670b07c3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:52.983203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd4ba8-7a13-4cc4-a417-9e5f670b07c3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:52.984546 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd4ba8-7a13-4cc4-a417-9e5f670b07c3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:52.993295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-84dd4ba8-7a13-4cc4-a417-9e5f670b07c3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 16:04:52.993295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:52.993295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 16:04:52.993295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 16:04:52.993295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 16:04:52.993295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 16:04:52.993295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 16:04:52.993295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:52.993295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 16:04:52.993295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:52.993295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 16:04:52.993295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:52.993295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 16:04:52.993295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 16:04:52.993295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 16:04:52.993295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 16:04:52.993295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 16:04:52.995330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 16:04:52.995330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 16:04:52.995330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 16:04:52.995330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 16:04:52.995330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 16:04:52.995330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 16:04:52.995330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 16:04:52.995330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 16:04:52.995330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 16:04:52.995330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 16:04:52.995330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 16:04:52.995330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 16:04:52.995330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 16:04:52.995330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 16:04:52.995330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 16:04:52.995330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 16:04:52.997354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 16:04:52.997354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Sep 06 16:04:52.997354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 16:04:52.997354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Sep 06 16:04:52.997354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 16:04:52.997354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 16:04:52.997354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 16:04:52.997354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Sep 06 16:04:52.997354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 16:04:52.997354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Sep 06 16:04:52.997354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 16:04:52.997354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 16:04:52.997354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 16:04:52.997354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Sep 06 16:04:52.997354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 655/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 6 16:04:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 16:04:53.002190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbbe86-c3aa-49da-b228-8980b50bf958 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:53.083280 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:04:53.109483 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbbe86-c3aa-49da-b228-8980b50bf958 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:53.145727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed257c4-f4f2-4c95-b9f1-08a92b1135ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:53.146337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed257c4-f4f2-4c95-b9f1-08a92b1135ef None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:53.146615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed257c4-f4f2-4c95-b9f1-08a92b1135ef None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:53.146923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed257c4-f4f2-4c95-b9f1-08a92b1135ef None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:53.147563 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed257c4-f4f2-4c95-b9f1-08a92b1135ef None admin] RBAC: Adding query filter params (name=placement) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:04:53.147865 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed257c4-f4f2-4c95-b9f1-08a92b1135ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:53.149244 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed257c4-f4f2-4c95-b9f1-08a92b1135ef None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:53.155265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 650/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Tue Sep 6 16:04:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:04:53.161321 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd4ba8-7a13-4cc4-a417-9e5f670b07c3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:04:53.267782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd4ba8-7a13-4cc4-a417-9e5f670b07c3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:04:53.307249 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6195a-9202-4cf2-b0f1-f82fd7972fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:04:53.308350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6195a-9202-4cf2-b0f1-f82fd7972fd1 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:04:53.308842 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6195a-9202-4cf2-b0f1-f82fd7972fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:04:53.309206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6195a-9202-4cf2-b0f1-f82fd7972fd1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:04:53.310530 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6195a-9202-4cf2-b0f1-f82fd7972fd1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:04:53.312437 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6195a-9202-4cf2-b0f1-f82fd7972fd1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:04:53.338828 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 656/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:04:53 2022] POST /identity/v3/endpoints => generated 331 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:05:03.156552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=30 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:05:03.334504 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:04.062853 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388f5c4-e40b-45b4-82e8-780601e78b93 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:04.063392 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388f5c4-e40b-45b4-82e8-780601e78b93 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:04.063392 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388f5c4-e40b-45b4-82e8-780601e78b93 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:04.065172 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 651/1307] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:04.070820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0e4d1-d0fd-4f87-9ed2-eb5551638216 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:04.071371 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0e4d1-d0fd-4f87-9ed2-eb5551638216 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:04.071371 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0e4d1-d0fd-4f87-9ed2-eb5551638216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:04.178021 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ffe0e4d1-d0fd-4f87-9ed2-eb5551638216 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:04.179354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ffe0e4d1-d0fd-4f87-9ed2-eb5551638216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:04.372895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 657/1308] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:06:04.691225 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6475259-24c0-4110-a710-c3a45abccc88 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:04.692410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6475259-24c0-4110-a710-c3a45abccc88 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:04.693376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6475259-24c0-4110-a710-c3a45abccc88 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:04.697523 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 652/1309] 10.222.0.26 () {54 vars in 965 bytes} [Tue Sep 6 16:06:04 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:04.706425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0a16c-21f5-42ca-b08b-b6606869d8f4 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:04.707835 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0a16c-21f5-42ca-b08b-b6606869d8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:04.709000 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0a16c-21f5-42ca-b08b-b6606869d8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:04.904430 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-39c0a16c-21f5-42ca-b08b-b6606869d8f4 None None] MFA Rules not processed for user `989beaca146d4f9b950f333bd3567e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:04.905773 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-39c0a16c-21f5-42ca-b08b-b6606869d8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:04.929365 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:04.992867 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 658/1310] 10.222.0.26 () {58 vars in 1062 bytes} [Tue Sep 6 16:06:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:06:05.004297 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6475259-24c0-4110-a710-c3a45abccc88 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:05.006813 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e6475259-24c0-4110-a710-c3a45abccc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:05.133437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6475259-24c0-4110-a710-c3a45abccc88 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:05.217166 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318e4e5-1263-4827-a8b4-4b97430e10c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:05.218156 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318e4e5-1263-4827-a8b4-4b97430e10c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:05.218558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318e4e5-1263-4827-a8b4-4b97430e10c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:05.218968 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318e4e5-1263-4827-a8b4-4b97430e10c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:05.221653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2318e4e5-1263-4827-a8b4-4b97430e10c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:05.286001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:05.353791 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318e4e5-1263-4827-a8b4-4b97430e10c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:05.355281 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318e4e5-1263-4827-a8b4-4b97430e10c1 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:05.375935 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 653/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:06:07.714863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612200e0-29ea-436e-9ce9-c89550f36a58 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:07.715507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612200e0-29ea-436e-9ce9-c89550f36a58 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:07.715507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612200e0-29ea-436e-9ce9-c89550f36a58 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:07.716772 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 659/1312] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:07.721995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245846a-e5cb-4ab2-b556-cdb92cc60fc3 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:07.722410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245846a-e5cb-4ab2-b556-cdb92cc60fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:07.722533 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245846a-e5cb-4ab2-b556-cdb92cc60fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:07.795292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0245846a-e5cb-4ab2-b556-cdb92cc60fc3 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:07.796528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0245846a-e5cb-4ab2-b556-cdb92cc60fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:07.895907 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 654/1313] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:06:08.188023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612200e0-29ea-436e-9ce9-c89550f36a58 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:08.190070 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-612200e0-29ea-436e-9ce9-c89550f36a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:08.295394 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612200e0-29ea-436e-9ce9-c89550f36a58 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:08.372108 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:08.374971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b107b-52ad-4993-b05e-39020a78d8d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:08.375765 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b107b-52ad-4993-b05e-39020a78d8d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:08.376206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b107b-52ad-4993-b05e-39020a78d8d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:08.376660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b107b-52ad-4993-b05e-39020a78d8d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:08.378933 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1c7b107b-52ad-4993-b05e-39020a78d8d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:08.490819 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b107b-52ad-4993-b05e-39020a78d8d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:08.492176 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b107b-52ad-4993-b05e-39020a78d8d5 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:08.510919 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 660/1314] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:06:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:06:10.770806 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6eb6b-182c-490a-8ca4-8168dbb5d7f3 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:10.771367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6eb6b-182c-490a-8ca4-8168dbb5d7f3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:10.771367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6eb6b-182c-490a-8ca4-8168dbb5d7f3 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:10.772690 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 655/1315] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:10.777997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcfd1f-c0e2-4729-9098-41501dcd54e1 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:10.778465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcfd1f-c0e2-4729-9098-41501dcd54e1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:10.778563 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcfd1f-c0e2-4729-9098-41501dcd54e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:10.847682 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-57fcfd1f-c0e2-4729-9098-41501dcd54e1 None None] MFA Rules not processed for user `5bbe86494fef439facc3d7e23e115ded`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:10.848889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-57fcfd1f-c0e2-4729-9098-41501dcd54e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:10.899466 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:10.945596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 661/1316] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:10 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:06:11.229237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6eb6b-182c-490a-8ca4-8168dbb5d7f3 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:11.230819 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-20e6eb6b-182c-490a-8ca4-8168dbb5d7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:11.241260 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:11.339688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6eb6b-182c-490a-8ca4-8168dbb5d7f3 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:11.413740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d7360-b41d-4bed-975a-db665d5568e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:11.414345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d7360-b41d-4bed-975a-db665d5568e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:11.414625 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d7360-b41d-4bed-975a-db665d5568e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:11.414936 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d7360-b41d-4bed-975a-db665d5568e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:11.416532 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-153d7360-b41d-4bed-975a-db665d5568e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:11.481041 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:11.527762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d7360-b41d-4bed-975a-db665d5568e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:11.529182 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d7360-b41d-4bed-975a-db665d5568e0 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:11.547437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 656/1317] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:06:11 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:06:14.109498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0394f-d108-4869-abf2-7fb862f41400 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:14.110077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0394f-d108-4869-abf2-7fb862f41400 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:14.110077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0394f-d108-4869-abf2-7fb862f41400 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:14.111478 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 662/1318] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:14.116873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2a418-a181-4eec-ba25-ca131ed17e32 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:14.117326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2a418-a181-4eec-ba25-ca131ed17e32 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:14.117424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2a418-a181-4eec-ba25-ca131ed17e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:14.186829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-94b2a418-a181-4eec-ba25-ca131ed17e32 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:14.188045 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-94b2a418-a181-4eec-ba25-ca131ed17e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:14.287951 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 657/1319] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:06:14.587850 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ab9ad-c367-44d7-8ecd-c03c50e06b01 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:14.588154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ab9ad-c367-44d7-8ecd-c03c50e06b01 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:14.588429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ab9ad-c367-44d7-8ecd-c03c50e06b01 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:14.589797 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 663/1320] 10.222.0.26 () {54 vars in 965 bytes} [Tue Sep 6 16:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:14.596557 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557106c-9f1a-45b2-ad25-7d81788f8a48 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:14.596877 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557106c-9f1a-45b2-ad25-7d81788f8a48 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:14.597154 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557106c-9f1a-45b2-ad25-7d81788f8a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:14.620422 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:14.682515 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1557106c-9f1a-45b2-ad25-7d81788f8a48 None None] MFA Rules not processed for user `989beaca146d4f9b950f333bd3567e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:14.683783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1557106c-9f1a-45b2-ad25-7d81788f8a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:14.766046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 658/1321] 10.222.0.26 () {58 vars in 1062 bytes} [Tue Sep 6 16:06:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:06:14.776677 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ab9ad-c367-44d7-8ecd-c03c50e06b01 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:14.779513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-654ab9ad-c367-44d7-8ecd-c03c50e06b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:14.906300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ab9ad-c367-44d7-8ecd-c03c50e06b01 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:14.992007 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d0656-d685-41ed-bd57-6722b477bb81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:14.992622 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d0656-d685-41ed-bd57-6722b477bb81 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:14.992897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d0656-d685-41ed-bd57-6722b477bb81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:14.993166 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d0656-d685-41ed-bd57-6722b477bb81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:14.994912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6c5d0656-d685-41ed-bd57-6722b477bb81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:15.003339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:15.115447 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d0656-d685-41ed-bd57-6722b477bb81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:15.116890 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d0656-d685-41ed-bd57-6722b477bb81 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:15.136337 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 664/1322] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:06:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:06:18.252496 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd105c1-029f-4ab1-9034-dba4b4287d31 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:18.253254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd105c1-029f-4ab1-9034-dba4b4287d31 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:18.253254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd105c1-029f-4ab1-9034-dba4b4287d31 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:18.254405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 659/1323] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:18.260059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd5a39-4786-4f0d-98a3-ad25049904dd None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:18.260691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd5a39-4786-4f0d-98a3-ad25049904dd None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:18.260691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd5a39-4786-4f0d-98a3-ad25049904dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:18.342226 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-cfdd5a39-4786-4f0d-98a3-ad25049904dd None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:18.343645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cfdd5a39-4786-4f0d-98a3-ad25049904dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:18.416141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:18.468410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 665/1324] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:06:18.751998 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd105c1-029f-4ab1-9034-dba4b4287d31 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:18.754245 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4bd105c1-029f-4ab1-9034-dba4b4287d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:18.873962 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd105c1-029f-4ab1-9034-dba4b4287d31 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:18.887776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:18.958630 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bca9ed-60a4-4d3c-a6f5-5ad6895e47e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:18.959274 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bca9ed-60a4-4d3c-a6f5-5ad6895e47e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:18.959550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bca9ed-60a4-4d3c-a6f5-5ad6895e47e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:18.959820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bca9ed-60a4-4d3c-a6f5-5ad6895e47e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:18.961410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a9bca9ed-60a4-4d3c-a6f5-5ad6895e47e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:19.085190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bca9ed-60a4-4d3c-a6f5-5ad6895e47e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:19.086634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bca9ed-60a4-4d3c-a6f5-5ad6895e47e4 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:19.106159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 660/1325] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:06:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:06:21.128543 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b09c4-3eba-4689-bea7-fbb0b1b4c1b5 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:21.129124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b09c4-3eba-4689-bea7-fbb0b1b4c1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:21.129124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b09c4-3eba-4689-bea7-fbb0b1b4c1b5 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:21.130470 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 666/1326] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:21.136117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd45ba-19a1-4e52-96b3-ce47265f6800 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:21.136612 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd45ba-19a1-4e52-96b3-ce47265f6800 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:21.136715 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd45ba-19a1-4e52-96b3-ce47265f6800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:21.188627 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:21.211026 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-cdbd45ba-19a1-4e52-96b3-ce47265f6800 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:21.213160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cdbd45ba-19a1-4e52-96b3-ce47265f6800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:21.340228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 661/1327] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:06:21.629707 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b09c4-3eba-4689-bea7-fbb0b1b4c1b5 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:21.631322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9e7b09c4-3eba-4689-bea7-fbb0b1b4c1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:21.757597 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b09c4-3eba-4689-bea7-fbb0b1b4c1b5 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:21.856364 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2980de-c11d-4d0c-b927-c79f90532b6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:21.857079 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2980de-c11d-4d0c-b927-c79f90532b6b service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:21.857418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2980de-c11d-4d0c-b927-c79f90532b6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:21.857750 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2980de-c11d-4d0c-b927-c79f90532b6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:21.859754 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0e2980de-c11d-4d0c-b927-c79f90532b6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:21.881018 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:22.030014 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2980de-c11d-4d0c-b927-c79f90532b6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:22.031522 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2980de-c11d-4d0c-b927-c79f90532b6b service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:22.056006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 667/1328] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:06:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:06:24.184103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285c002-78e6-4637-81af-150861763d1c None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:24.184662 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285c002-78e6-4637-81af-150861763d1c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:24.184662 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285c002-78e6-4637-81af-150861763d1c None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:24.186007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 662/1329] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:24.191869 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2523d34-3faf-4bcc-89de-3d18c80ed1a8 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:24.192349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2523d34-3faf-4bcc-89de-3d18c80ed1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:24.192465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2523d34-3faf-4bcc-89de-3d18c80ed1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:24.260301 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b2523d34-3faf-4bcc-89de-3d18c80ed1a8 None None] MFA Rules not processed for user `5bbe86494fef439facc3d7e23e115ded`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:24.261834 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b2523d34-3faf-4bcc-89de-3d18c80ed1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:24.332101 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:24.368212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 668/1330] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:24 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:06:24.655426 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285c002-78e6-4637-81af-150861763d1c None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:24.657031 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6285c002-78e6-4637-81af-150861763d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:24.776560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285c002-78e6-4637-81af-150861763d1c None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:24.800206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:24.876843 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15e53c-0841-44c7-8d82-5aecb7848804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:24.877454 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15e53c-0841-44c7-8d82-5aecb7848804 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:24.877734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15e53c-0841-44c7-8d82-5aecb7848804 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:24.877999 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15e53c-0841-44c7-8d82-5aecb7848804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:24.882728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1a15e53c-0841-44c7-8d82-5aecb7848804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:25.019069 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15e53c-0841-44c7-8d82-5aecb7848804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:25.020505 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15e53c-0841-44c7-8d82-5aecb7848804 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:25.039245 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 663/1331] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:06:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:06:27.225719 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb86491-b2e1-4c8b-a36c-f1aee4b5ecc5 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:27.226314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb86491-b2e1-4c8b-a36c-f1aee4b5ecc5 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:27.226314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb86491-b2e1-4c8b-a36c-f1aee4b5ecc5 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:27.227654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 669/1332] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:27.233222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685c6a6-dfbd-494c-9a4e-9d18c3f7f0be None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:27.233661 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685c6a6-dfbd-494c-9a4e-9d18c3f7f0be None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:27.233874 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685c6a6-dfbd-494c-9a4e-9d18c3f7f0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:27.301591 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:27.305048 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-5685c6a6-dfbd-494c-9a4e-9d18c3f7f0be None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:27.307253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5685c6a6-dfbd-494c-9a4e-9d18c3f7f0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:27.404565 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 664/1333] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:06:27.688371 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb86491-b2e1-4c8b-a36c-f1aee4b5ecc5 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:27.690617 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0fb86491-b2e1-4c8b-a36c-f1aee4b5ecc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:27.809891 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb86491-b2e1-4c8b-a36c-f1aee4b5ecc5 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:27.887259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbd531-90e6-41a2-bf83-160dadc61835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:27.887787 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbd531-90e6-41a2-bf83-160dadc61835 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:27.888063 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbd531-90e6-41a2-bf83-160dadc61835 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:27.888334 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbd531-90e6-41a2-bf83-160dadc61835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:27.890093 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0ebbd531-90e6-41a2-bf83-160dadc61835 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:27.924048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:28.025121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebbd531-90e6-41a2-bf83-160dadc61835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:28.026582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebbd531-90e6-41a2-bf83-160dadc61835 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:28.046968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 670/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:06:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:06:30.043121 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba8d7a-6e3e-4119-9545-d8a53c136890 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:30.043638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba8d7a-6e3e-4119-9545-d8a53c136890 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:30.043638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba8d7a-6e3e-4119-9545-d8a53c136890 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:30.044905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 665/1335] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:30.050830 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cec7b-9555-4d43-906e-32444883fabd None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:30.051253 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cec7b-9555-4d43-906e-32444883fabd None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:30.051365 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cec7b-9555-4d43-906e-32444883fabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:30.125667 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-531cec7b-9555-4d43-906e-32444883fabd None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:30.127551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-531cec7b-9555-4d43-906e-32444883fabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:30.210008 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:30.233979 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 671/1336] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:06:30.539546 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba8d7a-6e3e-4119-9545-d8a53c136890 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:30.541111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ddba8d7a-6e3e-4119-9545-d8a53c136890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:30.647311 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba8d7a-6e3e-4119-9545-d8a53c136890 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:30.678178 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:30.725329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c233a41-d119-467d-84c2-2a76ef38e2b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:30.725921 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c233a41-d119-467d-84c2-2a76ef38e2b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:30.726204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c233a41-d119-467d-84c2-2a76ef38e2b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:30.726475 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c233a41-d119-467d-84c2-2a76ef38e2b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:30.728103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2c233a41-d119-467d-84c2-2a76ef38e2b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:30.838956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c233a41-d119-467d-84c2-2a76ef38e2b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:30.840402 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c233a41-d119-467d-84c2-2a76ef38e2b3 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:30.859371 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 666/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:06:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:06:33.218512 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a1bed-3efc-4b07-8664-dbff0c2bdc6b None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:33.219080 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a1bed-3efc-4b07-8664-dbff0c2bdc6b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:33.219080 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a1bed-3efc-4b07-8664-dbff0c2bdc6b None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:33.220556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 672/1338] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:33.227064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa60cd-6a99-4360-9cd2-0dbbd22f8cc4 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:33.227588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa60cd-6a99-4360-9cd2-0dbbd22f8cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:33.227725 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa60cd-6a99-4360-9cd2-0dbbd22f8cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:33.304810 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1faa60cd-6a99-4360-9cd2-0dbbd22f8cc4 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:33.306038 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1faa60cd-6a99-4360-9cd2-0dbbd22f8cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:33.316189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:33.413855 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 667/1339] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:06:33.695627 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a1bed-3efc-4b07-8664-dbff0c2bdc6b None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:33.697181 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d07a1bed-3efc-4b07-8664-dbff0c2bdc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:33.810135 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a1bed-3efc-4b07-8664-dbff0c2bdc6b None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:33.891020 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1db60-9c86-4eec-9fda-3b03df099bf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:33.891516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1db60-9c86-4eec-9fda-3b03df099bf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:33.891796 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1db60-9c86-4eec-9fda-3b03df099bf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:33.892067 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1db60-9c86-4eec-9fda-3b03df099bf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:33.894344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3dd1db60-9c86-4eec-9fda-3b03df099bf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:33.933590 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:34.014249 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd1db60-9c86-4eec-9fda-3b03df099bf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:34.015735 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd1db60-9c86-4eec-9fda-3b03df099bf0 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:34.038042 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 673/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:06:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:06:38.288673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94aaed2-4b54-4685-93f4-de22c1c4ff99 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:38.289253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94aaed2-4b54-4685-93f4-de22c1c4ff99 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:38.289253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94aaed2-4b54-4685-93f4-de22c1c4ff99 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:38.290801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 668/1341] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:38.297727 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3100e-631c-44a6-b2e2-1a576cd3e186 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:38.298327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3100e-631c-44a6-b2e2-1a576cd3e186 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:38.298327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3100e-631c-44a6-b2e2-1a576cd3e186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:38.379012 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-57e3100e-631c-44a6-b2e2-1a576cd3e186 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:38.380442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-57e3100e-631c-44a6-b2e2-1a576cd3e186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:38.483498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:38.497328 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 674/1342] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:06:38.785279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94aaed2-4b54-4685-93f4-de22c1c4ff99 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:38.786856 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e94aaed2-4b54-4685-93f4-de22c1c4ff99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:38.911367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94aaed2-4b54-4685-93f4-de22c1c4ff99 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:38.964614 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:39.016434 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a9d03-b8b3-4b57-bc7b-fa533c030fdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:39.017051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a9d03-b8b3-4b57-bc7b-fa533c030fdb service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:39.017342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a9d03-b8b3-4b57-bc7b-fa533c030fdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:39.018486 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a9d03-b8b3-4b57-bc7b-fa533c030fdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:39.020191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-457a9d03-b8b3-4b57-bc7b-fa533c030fdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:39.150266 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a9d03-b8b3-4b57-bc7b-fa533c030fdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:39.151785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a9d03-b8b3-4b57-bc7b-fa533c030fdb service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:39.178897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 669/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:06:41.998544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6850486-84c6-47e0-b98b-d7ff3c70955c None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:41.999126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6850486-84c6-47e0-b98b-d7ff3c70955c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:41.999126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6850486-84c6-47e0-b98b-d7ff3c70955c None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:42.000464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 675/1344] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:42.006982 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01473c-f661-4d27-9d52-6cafbf5b12cc None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:42.007584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01473c-f661-4d27-9d52-6cafbf5b12cc None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:42.007584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01473c-f661-4d27-9d52-6cafbf5b12cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:42.084470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9e01473c-f661-4d27-9d52-6cafbf5b12cc None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:42.085695 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9e01473c-f661-4d27-9d52-6cafbf5b12cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:42.107123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:42.198492 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 670/1345] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:06:42.512192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6850486-84c6-47e0-b98b-d7ff3c70955c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:42.513957 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c6850486-84c6-47e0-b98b-d7ff3c70955c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:42.637093 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6850486-84c6-47e0-b98b-d7ff3c70955c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:42.723179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b69d2-99b6-4019-92b1-ae54a2f93104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:42.723787 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b69d2-99b6-4019-92b1-ae54a2f93104 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:42.724060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b69d2-99b6-4019-92b1-ae54a2f93104 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:42.724337 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b69d2-99b6-4019-92b1-ae54a2f93104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:42.725950 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ef1b69d2-99b6-4019-92b1-ae54a2f93104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:42.779582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:42.856799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b69d2-99b6-4019-92b1-ae54a2f93104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:42.858254 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b69d2-99b6-4019-92b1-ae54a2f93104 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:42.879438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 676/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:06:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:06:47.415270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fbe7e-5076-4e5a-863b-244873d6bf72 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:47.415770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fbe7e-5076-4e5a-863b-244873d6bf72 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:47.415874 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fbe7e-5076-4e5a-863b-244873d6bf72 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:47.417196 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 671/1347] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:47.423067 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1b081-895a-47cc-a0cb-bf5da0653996 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:47.424122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1b081-895a-47cc-a0cb-bf5da0653996 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:47.424122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1b081-895a-47cc-a0cb-bf5da0653996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:47.501132 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f5c1b081-895a-47cc-a0cb-bf5da0653996 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:47.502755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f5c1b081-895a-47cc-a0cb-bf5da0653996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:47.612565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:47.619050 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 677/1348] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:06:47.905084 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fbe7e-5076-4e5a-863b-244873d6bf72 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:06:47.906610 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-267fbe7e-5076-4e5a-863b-244873d6bf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:48.050275 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fbe7e-5076-4e5a-863b-244873d6bf72 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:06:48.103543 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:48.133407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2730ccc-e93d-4ee7-8ef9-4fdf2d67ec10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:06:48.134027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2730ccc-e93d-4ee7-8ef9-4fdf2d67ec10 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:48.134303 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2730ccc-e93d-4ee7-8ef9-4fdf2d67ec10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:48.134573 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2730ccc-e93d-4ee7-8ef9-4fdf2d67ec10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:48.136295 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f2730ccc-e93d-4ee7-8ef9-4fdf2d67ec10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:48.274918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2730ccc-e93d-4ee7-8ef9-4fdf2d67ec10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:06:48.276376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2730ccc-e93d-4ee7-8ef9-4fdf2d67ec10 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:06:48.297640 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 672/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:06:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:06:53.932196 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e90d0d-e756-45d6-a23c-a9f2bff81d5c None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:53.933381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e90d0d-e756-45d6-a23c-a9f2bff81d5c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:53.933381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e90d0d-e756-45d6-a23c-a9f2bff81d5c None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:53.938731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 678/1350] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 6 16:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:53.940808 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8311d9a-fe6b-4a9c-b2b1-b36c4bb7c93a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:53.941480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8311d9a-fe6b-4a9c-b2b1-b36c4bb7c93a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:53.941480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8311d9a-fe6b-4a9c-b2b1-b36c4bb7c93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:54.091606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-a8311d9a-fe6b-4a9c-b2b1-b36c4bb7c93a None None] MFA Rules not processed for user `034d75ab747e4c0cab040e47c6a6236e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:54.093511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a8311d9a-fe6b-4a9c-b2b1-b36c4bb7c93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:54.131675 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:54.343977 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 673/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 6 16:06:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:06:59.344479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613207c6-f15b-4734-a2a8-19362fbcba0f None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:59.345882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613207c6-f15b-4734-a2a8-19362fbcba0f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:59.346305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613207c6-f15b-4734-a2a8-19362fbcba0f None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:59.348004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 679/1352] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 6 16:06:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:06:59.353407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b9320-71e5-44d4-9ddf-df9c3f78949e None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:06:59.354092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b9320-71e5-44d4-9ddf-df9c3f78949e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:06:59.354092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b9320-71e5-44d4-9ddf-df9c3f78949e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:06:59.444861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9a6b9320-71e5-44d4-9ddf-df9c3f78949e None None] MFA Rules not processed for user `034d75ab747e4c0cab040e47c6a6236e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:06:59.446937 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9a6b9320-71e5-44d4-9ddf-df9c3f78949e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:06:59.581130 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:06:59.654733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 674/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 6 16:06:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:07:01.457651 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ae67c-20e3-4e63-883e-b79c24de815c None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:01.458248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ae67c-20e3-4e63-883e-b79c24de815c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:01.458248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ae67c-20e3-4e63-883e-b79c24de815c None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:01.459866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 680/1354] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 6 16:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:01.465047 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc59d4-21e7-4d4e-8510-edfd99b1f914 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:01.465817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc59d4-21e7-4d4e-8510-edfd99b1f914 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:01.465817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc59d4-21e7-4d4e-8510-edfd99b1f914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:01.559519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-eddc59d4-21e7-4d4e-8510-edfd99b1f914 None None] MFA Rules not processed for user `034d75ab747e4c0cab040e47c6a6236e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:01.560746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-eddc59d4-21e7-4d4e-8510-edfd99b1f914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:01.708942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 675/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 6 16:07:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:05.092785 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c29aa3-4aaa-424c-85d7-cb05e3bd9ddc None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:05.093467 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c29aa3-4aaa-424c-85d7-cb05e3bd9ddc None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:05.093467 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c29aa3-4aaa-424c-85d7-cb05e3bd9ddc None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:05.095900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 681/1356] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:05.102132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1653916-f6fe-4a73-9813-17e0ddf80ea7 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:05.102783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1653916-f6fe-4a73-9813-17e0ddf80ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:05.102783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1653916-f6fe-4a73-9813-17e0ddf80ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:05.148005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:05.192867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b1653916-f6fe-4a73-9813-17e0ddf80ea7 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:05.194085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b1653916-f6fe-4a73-9813-17e0ddf80ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:05.378759 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 676/1357] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:07:05.783645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf68af1-d8d6-4a1e-b89b-394c07ffae6d None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:05.784827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf68af1-d8d6-4a1e-b89b-394c07ffae6d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:05.785257 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf68af1-d8d6-4a1e-b89b-394c07ffae6d None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:05.787517 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 682/1358] 10.222.0.26 () {54 vars in 962 bytes} [Tue Sep 6 16:07:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:05.796325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9ffbb-7149-44cb-bb65-28bbb39e043b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:05.796621 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9ffbb-7149-44cb-bb65-28bbb39e043b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:05.796778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9ffbb-7149-44cb-bb65-28bbb39e043b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:05.912466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-62b9ffbb-7149-44cb-bb65-28bbb39e043b None None] MFA Rules not processed for user `6afd4f6ffdae4b16aa5ba757e68c18c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:05.913595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-62b9ffbb-7149-44cb-bb65-28bbb39e043b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:05.972173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:06.066424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 677/1359] 10.222.0.26 () {58 vars in 1059 bytes} [Tue Sep 6 16:07:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:06.074537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf68af1-d8d6-4a1e-b89b-394c07ffae6d None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:07:06.076080 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1cf68af1-d8d6-4a1e-b89b-394c07ffae6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:06.326753 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf68af1-d8d6-4a1e-b89b-394c07ffae6d None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:07:06.446810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:06.493719 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1f89a-7e9e-47c3-a7ae-81f392157ef8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:07:06.494423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1f89a-7e9e-47c3-a7ae-81f392157ef8 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:06.494633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1f89a-7e9e-47c3-a7ae-81f392157ef8 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:06.494920 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1f89a-7e9e-47c3-a7ae-81f392157ef8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:06.496518 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1db1f89a-7e9e-47c3-a7ae-81f392157ef8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:06.604026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1f89a-7e9e-47c3-a7ae-81f392157ef8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:07:06.605529 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1f89a-7e9e-47c3-a7ae-81f392157ef8 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:07:06.628725 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 683/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:07:09.371896 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05a9b6-907a-4ab9-b2fc-ceac1eb986b8 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:09.371896 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05a9b6-907a-4ab9-b2fc-ceac1eb986b8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:09.373207 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05a9b6-907a-4ab9-b2fc-ceac1eb986b8 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:09.380496 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e262d3-872f-43ce-bfbc-4d2a8fe8b76e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:09.380496 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e262d3-872f-43ce-bfbc-4d2a8fe8b76e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:09.380496 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e262d3-872f-43ce-bfbc-4d2a8fe8b76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:09.385744 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 678/1361] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:07:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:07:09.460590 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-44e262d3-872f-43ce-bfbc-4d2a8fe8b76e None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:09.461793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-44e262d3-872f-43ce-bfbc-4d2a8fe8b76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:09.528373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:09.604394 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 684/1362] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:07:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:09.925710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05a9b6-907a-4ab9-b2fc-ceac1eb986b8 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:07:09.927301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4c05a9b6-907a-4ab9-b2fc-ceac1eb986b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:10.239937 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05a9b6-907a-4ab9-b2fc-ceac1eb986b8 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:07:10.258901 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:10.437392 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98b6d2-4aca-4721-93d5-f1df66eafaa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:07:10.439400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98b6d2-4aca-4721-93d5-f1df66eafaa7 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:10.439925 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98b6d2-4aca-4721-93d5-f1df66eafaa7 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:10.440452 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98b6d2-4aca-4721-93d5-f1df66eafaa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:10.444316 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9e98b6d2-4aca-4721-93d5-f1df66eafaa7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:10.626856 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98b6d2-4aca-4721-93d5-f1df66eafaa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:07:10.629639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98b6d2-4aca-4721-93d5-f1df66eafaa7 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:07:10.668553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 679/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:07:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 743 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:07:13.073292 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238536f3-6d45-462b-a401-85fa9d8ca480 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:13.073943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238536f3-6d45-462b-a401-85fa9d8ca480 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:13.073943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238536f3-6d45-462b-a401-85fa9d8ca480 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:13.075479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 685/1364] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:13.081836 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb1fd6-dc27-4c9f-83ed-f3fd6a542656 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:13.082353 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb1fd6-dc27-4c9f-83ed-f3fd6a542656 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:13.082450 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb1fd6-dc27-4c9f-83ed-f3fd6a542656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:13.099526 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:13.157624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-39eb1fd6-dc27-4c9f-83ed-f3fd6a542656 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:13.158893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-39eb1fd6-dc27-4c9f-83ed-f3fd6a542656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:13.292339 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 680/1365] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:13.588443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c4cbf-e13e-4fa4-a67a-1b94a5a7b100 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:13.588976 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c4cbf-e13e-4fa4-a67a-1b94a5a7b100 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:13.590210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c4cbf-e13e-4fa4-a67a-1b94a5a7b100 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:13.592461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 686/1366] 10.222.0.26 () {54 vars in 962 bytes} [Tue Sep 6 16:07:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:13.602465 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624dc77-4dfc-4034-9978-93cab366135d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:13.602790 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624dc77-4dfc-4034-9978-93cab366135d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:13.603063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624dc77-4dfc-4034-9978-93cab366135d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:13.695274 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b624dc77-4dfc-4034-9978-93cab366135d None None] MFA Rules not processed for user `6afd4f6ffdae4b16aa5ba757e68c18c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:13.696727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b624dc77-4dfc-4034-9978-93cab366135d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:13.753133 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:13.870071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 681/1367] 10.222.0.26 () {58 vars in 1059 bytes} [Tue Sep 6 16:07:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:13.878653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c4cbf-e13e-4fa4-a67a-1b94a5a7b100 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:07:13.880338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ca0c4cbf-e13e-4fa4-a67a-1b94a5a7b100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:14.074256 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c4cbf-e13e-4fa4-a67a-1b94a5a7b100 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:07:14.108133 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:14.247734 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d60e7-ef66-46ce-97a2-d436c94b34f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:07:14.248768 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d60e7-ef66-46ce-97a2-d436c94b34f3 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:14.249329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d60e7-ef66-46ce-97a2-d436c94b34f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:14.249885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d60e7-ef66-46ce-97a2-d436c94b34f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:14.253554 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-271d60e7-ef66-46ce-97a2-d436c94b34f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:14.346778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8654c6f-dad7-4995-ac61-b04585f36548 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:14.347560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8654c6f-dad7-4995-ac61-b04585f36548 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:14.347560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8654c6f-dad7-4995-ac61-b04585f36548 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:14.348896 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 682/1368] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:07:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:07:14.355251 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b7e8a-40b4-4afb-bdf8-670106be1ce0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:14.355535 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b7e8a-40b4-4afb-bdf8-670106be1ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:14.355814 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b7e8a-40b4-4afb-bdf8-670106be1ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:14.396864 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d60e7-ef66-46ce-97a2-d436c94b34f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:07:14.400009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d60e7-ef66-46ce-97a2-d436c94b34f3 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:07:14.431901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 687/1369] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:07:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:07:14.444909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-708b7e8a-40b4-4afb-bdf8-670106be1ce0 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:14.446169 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-708b7e8a-40b4-4afb-bdf8-670106be1ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:14.587351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:14.609843 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 683/1370] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:07:15.293489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa52e5b-c381-4a95-897b-d6ccf7c4970c None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:15.294230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa52e5b-c381-4a95-897b-d6ccf7c4970c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:15.294955 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa52e5b-c381-4a95-897b-d6ccf7c4970c None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:15.298300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 688/1371] 10.222.0.26 () {54 vars in 964 bytes} [Tue Sep 6 16:07:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:15.309091 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd57e2e-b538-4bf1-a6a7-73e66ea13c22 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:15.309600 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd57e2e-b538-4bf1-a6a7-73e66ea13c22 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:15.310099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd57e2e-b538-4bf1-a6a7-73e66ea13c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:15.430358 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-6fd57e2e-b538-4bf1-a6a7-73e66ea13c22 None None] MFA Rules not processed for user `7849ef049ffd4495909f997735af8782`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:15.431661 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6fd57e2e-b538-4bf1-a6a7-73e66ea13c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:15.559789 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 684/1372] 10.222.0.26 () {58 vars in 1061 bytes} [Tue Sep 6 16:07:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:15.570579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa52e5b-c381-4a95-897b-d6ccf7c4970c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:07:15.572222 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-dfa52e5b-c381-4a95-897b-d6ccf7c4970c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:15.610802 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:15.712840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa52e5b-c381-4a95-897b-d6ccf7c4970c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:07:15.811975 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a14eb-7fbb-40d3-acaf-3b9fc191071b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:07:15.811975 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a14eb-7fbb-40d3-acaf-3b9fc191071b service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:15.814607 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a14eb-7fbb-40d3-acaf-3b9fc191071b service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:15.814607 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a14eb-7fbb-40d3-acaf-3b9fc191071b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:15.816770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-437a14eb-7fbb-40d3-acaf-3b9fc191071b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:15.915280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:15.959653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a14eb-7fbb-40d3-acaf-3b9fc191071b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:07:15.961519 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a14eb-7fbb-40d3-acaf-3b9fc191071b service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:07:15.984658 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 689/1373] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:07:16.646149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672ff14-9283-4a35-bcad-7ce9ece48f02 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:16.646779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672ff14-9283-4a35-bcad-7ce9ece48f02 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:16.646779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672ff14-9283-4a35-bcad-7ce9ece48f02 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:16.648087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 685/1374] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:16.654313 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83c177-237a-451d-885f-b035341a3cd2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:16.654904 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83c177-237a-451d-885f-b035341a3cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:16.655046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83c177-237a-451d-885f-b035341a3cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:16.742202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7d83c177-237a-451d-885f-b035341a3cd2 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:16.743505 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7d83c177-237a-451d-885f-b035341a3cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:16.867177 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 690/1375] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:17.170749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672ff14-9283-4a35-bcad-7ce9ece48f02 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:07:17.172322 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a672ff14-9283-4a35-bcad-7ce9ece48f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:17.310767 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:17.396477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672ff14-9283-4a35-bcad-7ce9ece48f02 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:07:17.531356 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd7c1f-5901-482f-b8c4-969b2fdfa94a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:07:17.532171 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd7c1f-5901-482f-b8c4-969b2fdfa94a service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:17.532488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd7c1f-5901-482f-b8c4-969b2fdfa94a service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:17.532658 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd7c1f-5901-482f-b8c4-969b2fdfa94a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:17.534237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bafd7c1f-5901-482f-b8c4-969b2fdfa94a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:17.596297 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:17.652677 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd7c1f-5901-482f-b8c4-969b2fdfa94a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:07:17.654359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd7c1f-5901-482f-b8c4-969b2fdfa94a service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:07:17.678825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 686/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:07:19.776776 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050a8c3-b8c3-440c-a60a-a16796194c9d None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:19.777552 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050a8c3-b8c3-440c-a60a-a16796194c9d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:19.777552 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050a8c3-b8c3-440c-a60a-a16796194c9d None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:19.778828 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 691/1377] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:19.786856 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48770b01-a7ad-4bfb-92cc-95082561afae None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:19.787423 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48770b01-a7ad-4bfb-92cc-95082561afae None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:19.787541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48770b01-a7ad-4bfb-92cc-95082561afae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:19.879548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-48770b01-a7ad-4bfb-92cc-95082561afae None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:19.880783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-48770b01-a7ad-4bfb-92cc-95082561afae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:20.007770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 687/1378] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:20.398950 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050a8c3-b8c3-440c-a60a-a16796194c9d None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:07:20.403611 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c050a8c3-b8c3-440c-a60a-a16796194c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:20.451597 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:20.626881 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050a8c3-b8c3-440c-a60a-a16796194c9d None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:07:20.785914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:20.792289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef648c5c-9744-4d62-85b6-9b73dbb51095 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:07:20.792905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef648c5c-9744-4d62-85b6-9b73dbb51095 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:20.793185 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef648c5c-9744-4d62-85b6-9b73dbb51095 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:20.793458 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef648c5c-9744-4d62-85b6-9b73dbb51095 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:20.795178 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ef648c5c-9744-4d62-85b6-9b73dbb51095 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:21.019278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef648c5c-9744-4d62-85b6-9b73dbb51095 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:07:21.021273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef648c5c-9744-4d62-85b6-9b73dbb51095 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:07:21.042282 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 692/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:07:23.286965 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14976f70-a2c5-40fe-aee6-ce7c4bd3b014 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:23.287376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14976f70-a2c5-40fe-aee6-ce7c4bd3b014 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:23.287525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14976f70-a2c5-40fe-aee6-ce7c4bd3b014 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:23.289416 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 688/1380] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:23.296333 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e134fbc-989e-47b6-8737-337b339e25c6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:23.296883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e134fbc-989e-47b6-8737-337b339e25c6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:23.297679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e134fbc-989e-47b6-8737-337b339e25c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:23.409284 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-5e134fbc-989e-47b6-8737-337b339e25c6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:23.410592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5e134fbc-989e-47b6-8737-337b339e25c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:23.488518 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:23.523811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 693/1381] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:23.823048 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14976f70-a2c5-40fe-aee6-ce7c4bd3b014 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:07:23.824568 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-14976f70-a2c5-40fe-aee6-ce7c4bd3b014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:23.856387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:24.004629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14976f70-a2c5-40fe-aee6-ce7c4bd3b014 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:07:24.115874 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:24.134499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa3da0-c424-41a7-a938-5077af007e8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:07:24.135063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa3da0-c424-41a7-a938-5077af007e8b service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:24.135343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa3da0-c424-41a7-a938-5077af007e8b service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:24.135601 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa3da0-c424-41a7-a938-5077af007e8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:24.137204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b1aa3da0-c424-41a7-a938-5077af007e8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:24.281029 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa3da0-c424-41a7-a938-5077af007e8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:07:24.283989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa3da0-c424-41a7-a938-5077af007e8b service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:07:24.317717 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 689/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:07:26.638617 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa714c-89dd-4d97-98a2-a564539073ea None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:26.639435 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa714c-89dd-4d97-98a2-a564539073ea None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:26.639435 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa714c-89dd-4d97-98a2-a564539073ea None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:26.640726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 694/1383] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:26.647571 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05130378-e827-4f77-bac8-42b7e70ad522 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:26.648739 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05130378-e827-4f77-bac8-42b7e70ad522 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:26.648739 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05130378-e827-4f77-bac8-42b7e70ad522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:26.738714 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-05130378-e827-4f77-bac8-42b7e70ad522 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:26.739938 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-05130378-e827-4f77-bac8-42b7e70ad522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:26.836573 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:26.906797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 690/1384] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:27.215379 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa714c-89dd-4d97-98a2-a564539073ea None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:07:27.216915 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-91aa714c-89dd-4d97-98a2-a564539073ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:27.416456 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa714c-89dd-4d97-98a2-a564539073ea None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:07:27.489989 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:27.572197 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82655884-9066-4149-a73c-5b233bdbce4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:07:27.572882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82655884-9066-4149-a73c-5b233bdbce4e service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:27.573192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82655884-9066-4149-a73c-5b233bdbce4e service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:27.573842 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82655884-9066-4149-a73c-5b233bdbce4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:27.575660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-82655884-9066-4149-a73c-5b233bdbce4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:27.691437 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82655884-9066-4149-a73c-5b233bdbce4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:07:27.693063 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82655884-9066-4149-a73c-5b233bdbce4e service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:07:27.714192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 695/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:07:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:07:29.991065 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d6ee6-f667-4886-ae2b-ea80d86aca30 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:29.991797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d6ee6-f667-4886-ae2b-ea80d86aca30 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:29.991797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d6ee6-f667-4886-ae2b-ea80d86aca30 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:29.993624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 691/1386] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:07:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:07:30.001443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e11ccc-34cd-4286-a76d-f99ed8a6f598 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:30.002091 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e11ccc-34cd-4286-a76d-f99ed8a6f598 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:30.002091 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e11ccc-34cd-4286-a76d-f99ed8a6f598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:30.083330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d7e11ccc-34cd-4286-a76d-f99ed8a6f598 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:30.084956 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d7e11ccc-34cd-4286-a76d-f99ed8a6f598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:30.098038 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:30.227420 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 696/1387] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:30.586943 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d6ee6-f667-4886-ae2b-ea80d86aca30 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:07:30.589208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-499d6ee6-f667-4886-ae2b-ea80d86aca30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:30.810087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d6ee6-f667-4886-ae2b-ea80d86aca30 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:07:30.905841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:31.020974 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be43ab-20a8-4758-ab8c-ac89d88c0667 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:07:31.021684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be43ab-20a8-4758-ab8c-ac89d88c0667 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:31.021844 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be43ab-20a8-4758-ab8c-ac89d88c0667 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:31.022276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be43ab-20a8-4758-ab8c-ac89d88c0667 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:31.023829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f3be43ab-20a8-4758-ab8c-ac89d88c0667 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:31.132102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be43ab-20a8-4758-ab8c-ac89d88c0667 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:07:31.133715 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be43ab-20a8-4758-ab8c-ac89d88c0667 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:07:31.154158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 692/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:07:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:07:32.529793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d7858-08fb-4d0a-b3ee-91667bebcc4d None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:32.530649 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d7858-08fb-4d0a-b3ee-91667bebcc4d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:32.530649 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d7858-08fb-4d0a-b3ee-91667bebcc4d None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:32.533663 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 697/1389] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:07:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:32.539280 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fdf15-be61-4fde-99f1-0bb1c84b14d6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:32.539738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fdf15-be61-4fde-99f1-0bb1c84b14d6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:32.539820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fdf15-be61-4fde-99f1-0bb1c84b14d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:32.618502 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:32.621967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b10fdf15-be61-4fde-99f1-0bb1c84b14d6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:32.624055 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b10fdf15-be61-4fde-99f1-0bb1c84b14d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:32.747424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 693/1390] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:33.059860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b131ab-9a76-4d41-afd8-855857182449 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:33.060124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b131ab-9a76-4d41-afd8-855857182449 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:33.060409 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b131ab-9a76-4d41-afd8-855857182449 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:33.061758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 698/1391] 10.222.0.26 () {54 vars in 961 bytes} [Tue Sep 6 16:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:33.067139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99add25-bc89-4f8c-bd79-6c71a3b28a76 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:33.067426 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99add25-bc89-4f8c-bd79-6c71a3b28a76 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:33.067703 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99add25-bc89-4f8c-bd79-6c71a3b28a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:33.174440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b99add25-bc89-4f8c-bd79-6c71a3b28a76 None None] MFA Rules not processed for user `e50ef7db27744c7ebd2ec95cecc9d1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:33.176707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b99add25-bc89-4f8c-bd79-6c71a3b28a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:33.277935 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:33.286278 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 694/1392] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Sep 6 16:07:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:33.293597 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e267211-fe8c-457c-b621-83c124d9fbfa None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:33.294261 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e267211-fe8c-457c-b621-83c124d9fbfa None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:33.294721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e267211-fe8c-457c-b621-83c124d9fbfa None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:33.295813 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99add25-bc89-4f8c-bd79-6c71a3b28a76 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:07:33.297252 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 699/1393] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:07:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:33.304565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e0aa4-fa89-4002-9bb7-687bb14350a1 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:33.304847 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e0aa4-fa89-4002-9bb7-687bb14350a1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:33.305115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e0aa4-fa89-4002-9bb7-687bb14350a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:33.308464 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99add25-bc89-4f8c-bd79-6c71a3b28a76 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:07:33.388313 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e88e0aa4-fa89-4002-9bb7-687bb14350a1 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:33.389542 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e88e0aa4-fa89-4002-9bb7-687bb14350a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:33.395819 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01fd81-c545-4809-bee0-5eb99cce3796 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50ef7db27744c7ebd2ec95cecc9d1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:07:33.396437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01fd81-c545-4809-bee0-5eb99cce3796 service glance] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:33.396713 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01fd81-c545-4809-bee0-5eb99cce3796 service glance] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:33.396988 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01fd81-c545-4809-bee0-5eb99cce3796 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:33.398627 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5a01fd81-c545-4809-bee0-5eb99cce3796 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:33.498344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:33.512388 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 700/1394] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:07:33.550259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01fd81-c545-4809-bee0-5eb99cce3796 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:07:33.552238 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01fd81-c545-4809-bee0-5eb99cce3796 service glance] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:07:33.581481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 695/1395] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:07:33.805233 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e0aa4-fa89-4002-9bb7-687bb14350a1 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:07:33.806470 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e88e0aa4-fa89-4002-9bb7-687bb14350a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:33.964067 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e0aa4-fa89-4002-9bb7-687bb14350a1 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:07:34.112214 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:34.138701 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b8191-507a-4bb8-aba6-85ec14fbe93e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:07:34.139317 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b8191-507a-4bb8-aba6-85ec14fbe93e service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:34.139612 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b8191-507a-4bb8-aba6-85ec14fbe93e service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:34.139857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b8191-507a-4bb8-aba6-85ec14fbe93e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:34.142517 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d82b8191-507a-4bb8-aba6-85ec14fbe93e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:34.268141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b8191-507a-4bb8-aba6-85ec14fbe93e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:07:34.269763 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b8191-507a-4bb8-aba6-85ec14fbe93e service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:07:34.291515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 701/1396] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:07:34.400330 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fead737-b949-4e57-b153-e277e3a39b70 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:34.400616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fead737-b949-4e57-b153-e277e3a39b70 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:34.400891 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fead737-b949-4e57-b153-e277e3a39b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:34.447382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:34.527613 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1fead737-b949-4e57-b153-e277e3a39b70 None None] MFA Rules not processed for user `ba58a4914ddf418db9e8b9dce5784cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:34.529322 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1fead737-b949-4e57-b153-e277e3a39b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:34.648267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 696/1397] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 6 16:07:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:07:34.710232 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cb3d5-d355-46df-9a18-832ff1b37b10 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:34.710853 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cb3d5-d355-46df-9a18-832ff1b37b10 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:34.711224 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cb3d5-d355-46df-9a18-832ff1b37b10 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:34.713497 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 702/1398] 10.222.0.26 () {54 vars in 943 bytes} [Tue Sep 6 16:07:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:34.722494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4d4ef-a040-4629-8816-e51278f8231c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:34.722892 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4d4ef-a040-4629-8816-e51278f8231c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:34.723227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4d4ef-a040-4629-8816-e51278f8231c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:34.818704 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-41b4d4ef-a040-4629-8816-e51278f8231c None None] MFA Rules not processed for user `ece176d412574790a13d03dcca08f91a`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:34.820060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-41b4d4ef-a040-4629-8816-e51278f8231c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:34.904592 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:35.019110 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 697/1399] 10.222.0.26 () {58 vars in 1040 bytes} [Tue Sep 6 16:07:34 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:35.030500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9cb3d5-d355-46df-9a18-832ff1b37b10 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:07:35.033895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7c9cb3d5-d355-46df-9a18-832ff1b37b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:35.191033 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:35.229726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9cb3d5-d355-46df-9a18-832ff1b37b10 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:07:35.386275 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713792f-0b80-47cc-a9bc-2d2ade1f0560 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ece176d412574790a13d03dcca08f91a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:07:35.387308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713792f-0b80-47cc-a9bc-2d2ade1f0560 service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:35.387807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713792f-0b80-47cc-a9bc-2d2ade1f0560 service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:35.388270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713792f-0b80-47cc-a9bc-2d2ade1f0560 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:35.392283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2713792f-0b80-47cc-a9bc-2d2ade1f0560 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:35.538100 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713792f-0b80-47cc-a9bc-2d2ade1f0560 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:07:35.540391 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713792f-0b80-47cc-a9bc-2d2ade1f0560 service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:07:35.551547 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 703/1400] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 6 16:07:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:07:36.449955 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55981dd2-d5c0-43e1-a2d8-0e677ca9553f None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:36.450528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55981dd2-d5c0-43e1-a2d8-0e677ca9553f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:36.450528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55981dd2-d5c0-43e1-a2d8-0e677ca9553f None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:36.452039 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 698/1401] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:36.459051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb8533-d64e-41a1-936b-c87b22ad61ea None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:36.462095 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb8533-d64e-41a1-936b-c87b22ad61ea None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:36.462558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb8533-d64e-41a1-936b-c87b22ad61ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:36.474970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:36.562898 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-33eb8533-d64e-41a1-936b-c87b22ad61ea None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:36.566473 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-33eb8533-d64e-41a1-936b-c87b22ad61ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:36.741478 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 704/1402] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:37.060953 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55981dd2-d5c0-43e1-a2d8-0e677ca9553f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:07:37.062903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-55981dd2-d5c0-43e1-a2d8-0e677ca9553f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:37.339698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55981dd2-d5c0-43e1-a2d8-0e677ca9553f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:07:37.370706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:37.503793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d51f9-2ba3-4f44-8f56-2ed7a97dd59f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:07:37.504869 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d51f9-2ba3-4f44-8f56-2ed7a97dd59f service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:37.504869 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d51f9-2ba3-4f44-8f56-2ed7a97dd59f service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:37.505327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d51f9-2ba3-4f44-8f56-2ed7a97dd59f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:37.506640 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4c3d51f9-2ba3-4f44-8f56-2ed7a97dd59f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:37.659935 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d51f9-2ba3-4f44-8f56-2ed7a97dd59f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:07:37.662863 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d51f9-2ba3-4f44-8f56-2ed7a97dd59f service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:07:37.706811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 699/1403] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:07:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:07:39.958829 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e1278-fb25-4c8e-9813-4ee100839a50 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:39.959399 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e1278-fb25-4c8e-9813-4ee100839a50 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:39.959399 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e1278-fb25-4c8e-9813-4ee100839a50 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:39.961054 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 705/1404] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:39.966002 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c90fc1-ad4d-4083-a925-74a442b99d80 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:39.966992 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c90fc1-ad4d-4083-a925-74a442b99d80 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:39.966992 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c90fc1-ad4d-4083-a925-74a442b99d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:40.012862 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:40.073752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-38c90fc1-ad4d-4083-a925-74a442b99d80 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:40.075137 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-38c90fc1-ad4d-4083-a925-74a442b99d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:40.220589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 700/1405] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:40.550091 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e1278-fb25-4c8e-9813-4ee100839a50 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:07:40.553665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-785e1278-fb25-4c8e-9813-4ee100839a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:40.736629 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:40.850137 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e1278-fb25-4c8e-9813-4ee100839a50 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:07:41.013594 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00243fc6-451a-4623-9bed-2599e820e414 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:07:41.014939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00243fc6-451a-4623-9bed-2599e820e414 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:41.014939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00243fc6-451a-4623-9bed-2599e820e414 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:41.014939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00243fc6-451a-4623-9bed-2599e820e414 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:41.016506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-00243fc6-451a-4623-9bed-2599e820e414 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:41.076420 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:41.130602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00243fc6-451a-4623-9bed-2599e820e414 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:07:41.136748 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00243fc6-451a-4623-9bed-2599e820e414 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:07:41.158283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 706/1406] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:07:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:07:43.228464 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80583ba-cf11-4bc9-b168-52fd638d33a7 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:43.229042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80583ba-cf11-4bc9-b168-52fd638d33a7 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:43.229042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80583ba-cf11-4bc9-b168-52fd638d33a7 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:43.231811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 701/1407] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:07:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:43.239164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3c0d3-e84f-4bdc-823f-f77b81c9c2b0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:43.239733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3c0d3-e84f-4bdc-823f-f77b81c9c2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:43.239733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3c0d3-e84f-4bdc-823f-f77b81c9c2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:43.348181 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-01a3c0d3-e84f-4bdc-823f-f77b81c9c2b0 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:43.349424 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-01a3c0d3-e84f-4bdc-823f-f77b81c9c2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:43.491944 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 707/1408] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:43.910315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80583ba-cf11-4bc9-b168-52fd638d33a7 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:07:43.911936 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c80583ba-cf11-4bc9-b168-52fd638d33a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:44.067167 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:44.133044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80583ba-cf11-4bc9-b168-52fd638d33a7 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:07:44.281625 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d3390-a72c-4fca-adbc-5382587bc09f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:07:44.282292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d3390-a72c-4fca-adbc-5382587bc09f service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:44.282427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d3390-a72c-4fca-adbc-5382587bc09f service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:44.282731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d3390-a72c-4fca-adbc-5382587bc09f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:44.284389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-250d3390-a72c-4fca-adbc-5382587bc09f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:44.400189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:44.429947 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d3390-a72c-4fca-adbc-5382587bc09f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:07:44.431757 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d3390-a72c-4fca-adbc-5382587bc09f service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:07:44.459685 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 702/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:07:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:07:53.489876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=88 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:54.455772 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=14 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:58.594045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63520e3-00cf-4674-9668-2000f7f96540 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:58.594938 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63520e3-00cf-4674-9668-2000f7f96540 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:58.595121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63520e3-00cf-4674-9668-2000f7f96540 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:58.596774 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 708/1410] 10.222.0.26 () {54 vars in 918 bytes} [Tue Sep 6 16:07:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:58.603371 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6424653-e09e-4a06-8d85-51922462ba2d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:58.604062 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6424653-e09e-4a06-8d85-51922462ba2d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:58.604062 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6424653-e09e-4a06-8d85-51922462ba2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:58.682832 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e6424653-e09e-4a06-8d85-51922462ba2d None None] MFA Rules not processed for user `034d75ab747e4c0cab040e47c6a6236e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:58.684045 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e6424653-e09e-4a06-8d85-51922462ba2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:58.802229 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 703/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Tue Sep 6 16:07:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:07:58.817966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c2e11-576e-464e-aed7-c08edb055141 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:58.818250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c2e11-576e-464e-aed7-c08edb055141 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:58.818527 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c2e11-576e-464e-aed7-c08edb055141 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:58.819964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 709/1412] 10.222.0.26 () {54 vars in 918 bytes} [Tue Sep 6 16:07:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:07:58.825270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6af664-5d88-4b71-87e9-dd50f493698b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:07:58.825450 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6af664-5d88-4b71-87e9-dd50f493698b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:07:58.825663 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6af664-5d88-4b71-87e9-dd50f493698b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:07:58.907178 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:07:58.910680 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1d6af664-5d88-4b71-87e9-dd50f493698b None None] MFA Rules not processed for user `6afd4f6ffdae4b16aa5ba757e68c18c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:07:58.912594 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1d6af664-5d88-4b71-87e9-dd50f493698b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:07:59.043906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 704/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Tue Sep 6 16:07:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:01.451084 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5c4e2-16a1-40cb-9b7e-afe858a63682 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:01.451666 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5c4e2-16a1-40cb-9b7e-afe858a63682 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:01.451666 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5c4e2-16a1-40cb-9b7e-afe858a63682 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:01.453609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:08:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:01.459387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d931d99-7e9f-4c8e-bed5-9f2244f676e4 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:01.460102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d931d99-7e9f-4c8e-bed5-9f2244f676e4 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:01.460102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d931d99-7e9f-4c8e-bed5-9f2244f676e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:01.531719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-6d931d99-7e9f-4c8e-bed5-9f2244f676e4 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:01.532974 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6d931d99-7e9f-4c8e-bed5-9f2244f676e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:01.595945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:01.642585 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:01.949664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5c4e2-16a1-40cb-9b7e-afe858a63682 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:01.951291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-80e5c4e2-16a1-40cb-9b7e-afe858a63682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:02.109819 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5c4e2-16a1-40cb-9b7e-afe858a63682 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:02.186725 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:02.230446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fa398-f5c3-419b-8798-94c06b4b9ec6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:02.231079 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fa398-f5c3-419b-8798-94c06b4b9ec6 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:02.231352 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fa398-f5c3-419b-8798-94c06b4b9ec6 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:02.231618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fa398-f5c3-419b-8798-94c06b4b9ec6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:02.233216 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e81fa398-f5c3-419b-8798-94c06b4b9ec6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:02.329496 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fa398-f5c3-419b-8798-94c06b4b9ec6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:02.331119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fa398-f5c3-419b-8798-94c06b4b9ec6 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:02.351216 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 711/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:03.288903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628fd90-4acb-4843-b7ee-bd2d6358be7b None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:03.289468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628fd90-4acb-4843-b7ee-bd2d6358be7b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:03.289585 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628fd90-4acb-4843-b7ee-bd2d6358be7b None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:03.291013 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:03.296478 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc686c20-1bc1-484b-9bb8-eaa33b75e201 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:03.297090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc686c20-1bc1-484b-9bb8-eaa33b75e201 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:03.297090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc686c20-1bc1-484b-9bb8-eaa33b75e201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:03.375626 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-fc686c20-1bc1-484b-9bb8-eaa33b75e201 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:03.376841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fc686c20-1bc1-484b-9bb8-eaa33b75e201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:03.405044 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:03.481811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:03.632426 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628fd90-4acb-4843-b7ee-bd2d6358be7b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:03.633942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3628fd90-4acb-4843-b7ee-bd2d6358be7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:03.789561 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628fd90-4acb-4843-b7ee-bd2d6358be7b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:03.831842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:03.917943 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2047a-9313-476f-bb3b-b31113d5d2d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:03.918570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2047a-9313-476f-bb3b-b31113d5d2d0 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:03.918870 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2047a-9313-476f-bb3b-b31113d5d2d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:03.919758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2047a-9313-476f-bb3b-b31113d5d2d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:03.920775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a6c2047a-9313-476f-bb3b-b31113d5d2d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:04.014468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2047a-9313-476f-bb3b-b31113d5d2d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:04.016150 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2047a-9313-476f-bb3b-b31113d5d2d0 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:04.035201 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 707/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:04.924466 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a002cc-3fd0-4c83-9b14-7d18ed2e7e7d None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:04.924947 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a002cc-3fd0-4c83-9b14-7d18ed2e7e7d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:04.925071 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a002cc-3fd0-4c83-9b14-7d18ed2e7e7d None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:04.926367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:04.931590 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44e6df-da80-40a6-80c3-57760e023479 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:04.932063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44e6df-da80-40a6-80c3-57760e023479 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:04.932410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44e6df-da80-40a6-80c3-57760e023479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:04.989532 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:05.012463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-bc44e6df-da80-40a6-80c3-57760e023479 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:05.013684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bc44e6df-da80-40a6-80c3-57760e023479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:05.128348 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:05.284214 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a002cc-3fd0-4c83-9b14-7d18ed2e7e7d None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:05.285813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-20a002cc-3fd0-4c83-9b14-7d18ed2e7e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:05.444774 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a002cc-3fd0-4c83-9b14-7d18ed2e7e7d None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:05.459222 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:05.573642 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31852651-5640-42e0-aa19-e37cc3fd2634 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:05.574164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31852651-5640-42e0-aa19-e37cc3fd2634 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:05.574449 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31852651-5640-42e0-aa19-e37cc3fd2634 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:05.574756 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31852651-5640-42e0-aa19-e37cc3fd2634 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:05.576370 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-31852651-5640-42e0-aa19-e37cc3fd2634 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:05.669197 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31852651-5640-42e0-aa19-e37cc3fd2634 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:05.670850 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31852651-5640-42e0-aa19-e37cc3fd2634 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:05.689854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 714/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:06.615439 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57ce75-50d2-4540-a746-5664e1f14827 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:06.615952 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57ce75-50d2-4540-a746-5664e1f14827 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:06.615952 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57ce75-50d2-4540-a746-5664e1f14827 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:06.617584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:08:06.625304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055eb66-ca75-4355-827f-82bc93e86d3d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:06.625795 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055eb66-ca75-4355-827f-82bc93e86d3d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:06.625903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055eb66-ca75-4355-827f-82bc93e86d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:06.646394 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:06.711835 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4055eb66-ca75-4355-827f-82bc93e86d3d None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:06.713059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4055eb66-ca75-4355-827f-82bc93e86d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:06.823430 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:06.980695 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57ce75-50d2-4540-a746-5664e1f14827 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:06.982270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6e57ce75-50d2-4540-a746-5664e1f14827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:07.104643 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:07.143139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57ce75-50d2-4540-a746-5664e1f14827 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:07.263903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5eb35-9173-4202-a875-0b4fe27a40d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:07.264416 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5eb35-9173-4202-a875-0b4fe27a40d2 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:07.265249 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5eb35-9173-4202-a875-0b4fe27a40d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:07.265659 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5eb35-9173-4202-a875-0b4fe27a40d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:07.267274 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6ec5eb35-9173-4202-a875-0b4fe27a40d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:07.344402 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:07.358780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5eb35-9173-4202-a875-0b4fe27a40d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:07.360381 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5eb35-9173-4202-a875-0b4fe27a40d2 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:07.379065 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 710/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:08.252361 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85488eb6-395f-4a90-865e-aa03c2a5eded None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:08.252942 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85488eb6-395f-4a90-865e-aa03c2a5eded None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:08.252942 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85488eb6-395f-4a90-865e-aa03c2a5eded None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:08.254297 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:08:08.259728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad82e2-a876-439a-a4f2-547b06a11730 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:08.260232 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad82e2-a876-439a-a4f2-547b06a11730 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:08.260341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad82e2-a876-439a-a4f2-547b06a11730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:08.346939 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e9ad82e2-a876-439a-a4f2-547b06a11730 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:08.348148 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e9ad82e2-a876-439a-a4f2-547b06a11730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:08.447104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:08.617050 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85488eb6-395f-4a90-865e-aa03c2a5eded None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:08.618569 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-85488eb6-395f-4a90-865e-aa03c2a5eded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:08.706690 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:08.771588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85488eb6-395f-4a90-865e-aa03c2a5eded None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:08.886165 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d95cf-c25f-4346-ada2-607d55396d78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:08.886713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d95cf-c25f-4346-ada2-607d55396d78 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:08.886998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d95cf-c25f-4346-ada2-607d55396d78 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:08.887270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d95cf-c25f-4346-ada2-607d55396d78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:08.888908 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f55d95cf-c25f-4346-ada2-607d55396d78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:08.948205 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:08.993065 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d95cf-c25f-4346-ada2-607d55396d78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:08.994680 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d95cf-c25f-4346-ada2-607d55396d78 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:09.013523 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 717/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:09.829213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d3221-f844-4f46-a90e-23e957d508f1 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:09.829868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d3221-f844-4f46-a90e-23e957d508f1 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:09.829868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d3221-f844-4f46-a90e-23e957d508f1 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:09.831162 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:08:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:09.836451 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61fe97-cc63-4444-bb3b-83e5a5f486bc None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:09.836934 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61fe97-cc63-4444-bb3b-83e5a5f486bc None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:09.837039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61fe97-cc63-4444-bb3b-83e5a5f486bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:09.917598 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ed61fe97-cc63-4444-bb3b-83e5a5f486bc None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:09.918748 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ed61fe97-cc63-4444-bb3b-83e5a5f486bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:10.023108 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:10.177132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d3221-f844-4f46-a90e-23e957d508f1 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:10.178548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-168d3221-f844-4f46-a90e-23e957d508f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:10.232936 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:10.339232 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d3221-f844-4f46-a90e-23e957d508f1 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:10.457332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd471b-a626-4aeb-a582-ce07df4409e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:10.457851 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd471b-a626-4aeb-a582-ce07df4409e2 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:10.458123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd471b-a626-4aeb-a582-ce07df4409e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:10.458388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd471b-a626-4aeb-a582-ce07df4409e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:10.460015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3fdd471b-a626-4aeb-a582-ce07df4409e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:10.467141 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:10.551459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd471b-a626-4aeb-a582-ce07df4409e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:10.553056 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd471b-a626-4aeb-a582-ce07df4409e2 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:10.571712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 713/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:11.394065 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d996a0-1bfd-467a-b6f9-702674132f69 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:11.394634 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d996a0-1bfd-467a-b6f9-702674132f69 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:11.394634 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d996a0-1bfd-467a-b6f9-702674132f69 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:11.396002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:11.401151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bb5df-cfd1-4da8-ab05-2146d8a2a8a6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:11.401560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bb5df-cfd1-4da8-ab05-2146d8a2a8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:11.401797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bb5df-cfd1-4da8-ab05-2146d8a2a8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:11.479460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-ff0bb5df-cfd1-4da8-ab05-2146d8a2a8a6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:11.480682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ff0bb5df-cfd1-4da8-ab05-2146d8a2a8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:11.550107 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:11.581606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:11.744950 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d996a0-1bfd-467a-b6f9-702674132f69 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:11.746465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c7d996a0-1bfd-467a-b6f9-702674132f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:11.764961 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:11.899516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d996a0-1bfd-467a-b6f9-702674132f69 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:12.008109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:12.039743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f29ee2-b784-484d-8ebe-f4817c75d873 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:12.040347 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f29ee2-b784-484d-8ebe-f4817c75d873 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:12.040689 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f29ee2-b784-484d-8ebe-f4817c75d873 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:12.040966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f29ee2-b784-484d-8ebe-f4817c75d873 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:12.042593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-50f29ee2-b784-484d-8ebe-f4817c75d873 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:12.138578 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f29ee2-b784-484d-8ebe-f4817c75d873 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:12.140761 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f29ee2-b784-484d-8ebe-f4817c75d873 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:12.160065 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 720/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:12.981712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf46e86-cac5-47a4-8a81-4cc283dca6e3 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:12.982258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf46e86-cac5-47a4-8a81-4cc283dca6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:12.982258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf46e86-cac5-47a4-8a81-4cc283dca6e3 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:12.983958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:12.989733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8028d03-e6be-43ba-8969-b7ef2eef2d66 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:12.990359 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8028d03-e6be-43ba-8969-b7ef2eef2d66 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:12.990359 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8028d03-e6be-43ba-8969-b7ef2eef2d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:13.079268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a8028d03-e6be-43ba-8969-b7ef2eef2d66 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:13.080511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a8028d03-e6be-43ba-8969-b7ef2eef2d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:13.126911 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:13.183626 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:13.336555 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf46e86-cac5-47a4-8a81-4cc283dca6e3 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:13.338090 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8bf46e86-cac5-47a4-8a81-4cc283dca6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:13.494681 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf46e86-cac5-47a4-8a81-4cc283dca6e3 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:13.549384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:13.611963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113bf94-e303-45b2-ab87-18bd1c0e5bde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:13.612589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113bf94-e303-45b2-ab87-18bd1c0e5bde service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:13.612867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113bf94-e303-45b2-ab87-18bd1c0e5bde service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:13.613138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113bf94-e303-45b2-ab87-18bd1c0e5bde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:13.614809 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4113bf94-e303-45b2-ab87-18bd1c0e5bde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:13.706694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113bf94-e303-45b2-ab87-18bd1c0e5bde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:13.708446 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113bf94-e303-45b2-ab87-18bd1c0e5bde service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:13.727165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 716/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:14.537150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6fe49-2765-4a48-99e4-39333d4dddfd None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:14.537726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6fe49-2765-4a48-99e4-39333d4dddfd None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:14.537726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6fe49-2765-4a48-99e4-39333d4dddfd None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:14.539059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:14.543994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6e887-725e-4ff2-a073-63ae158510d0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:14.544411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6e887-725e-4ff2-a073-63ae158510d0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:14.544517 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6e887-725e-4ff2-a073-63ae158510d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:14.623837 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-24a6e887-725e-4ff2-a073-63ae158510d0 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:14.625066 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-24a6e887-725e-4ff2-a073-63ae158510d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:14.627223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:14.725118 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:14.878178 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6fe49-2765-4a48-99e4-39333d4dddfd None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:14.879737 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-43d6fe49-2765-4a48-99e4-39333d4dddfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:15.037874 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6fe49-2765-4a48-99e4-39333d4dddfd None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:15.067727 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:15.158147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f5544-a2ed-4b4e-a604-ec90e99cc389 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:15.158671 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f5544-a2ed-4b4e-a604-ec90e99cc389 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:15.158965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f5544-a2ed-4b4e-a604-ec90e99cc389 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:15.159237 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f5544-a2ed-4b4e-a604-ec90e99cc389 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:15.160847 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-339f5544-a2ed-4b4e-a604-ec90e99cc389 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:15.252204 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f5544-a2ed-4b4e-a604-ec90e99cc389 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:15.253785 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f5544-a2ed-4b4e-a604-ec90e99cc389 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:15.272014 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 723/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:17.175650 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766e924-de6b-47dc-aadf-ff9dd41da369 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:17.176207 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766e924-de6b-47dc-aadf-ff9dd41da369 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:17.176207 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766e924-de6b-47dc-aadf-ff9dd41da369 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:17.177573 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:17.183170 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34626c-3779-4969-b02e-7fd945773e67 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:17.183640 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34626c-3779-4969-b02e-7fd945773e67 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:17.183747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34626c-3779-4969-b02e-7fd945773e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:17.226267 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:17.254203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2b34626c-3779-4969-b02e-7fd945773e67 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:17.255474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2b34626c-3779-4969-b02e-7fd945773e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:17.356415 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:17.651377 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766e924-de6b-47dc-aadf-ff9dd41da369 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:17.652901 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d766e924-de6b-47dc-aadf-ff9dd41da369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:17.768541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766e924-de6b-47dc-aadf-ff9dd41da369 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:17.802110 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:17.845786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc343274-d049-4956-9316-fb1bbd6dcf28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:17.846393 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc343274-d049-4956-9316-fb1bbd6dcf28 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:17.846701 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc343274-d049-4956-9316-fb1bbd6dcf28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:17.846964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc343274-d049-4956-9316-fb1bbd6dcf28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:17.848547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cc343274-d049-4956-9316-fb1bbd6dcf28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:17.960250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc343274-d049-4956-9316-fb1bbd6dcf28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:17.961521 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc343274-d049-4956-9316-fb1bbd6dcf28 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:17.981857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 719/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:20.162800 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae488aa6-63da-477b-bd16-0b81111b3566 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:20.163376 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae488aa6-63da-477b-bd16-0b81111b3566 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:20.163376 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae488aa6-63da-477b-bd16-0b81111b3566 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:20.164757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:20.170270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088a474-4142-418a-a39a-0331dede797f None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:20.171020 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088a474-4142-418a-a39a-0331dede797f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:20.171020 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088a474-4142-418a-a39a-0331dede797f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:20.239450 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-2088a474-4142-418a-a39a-0331dede797f None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:20.240669 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2088a474-4142-418a-a39a-0331dede797f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:20.254335 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:20.346557 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:20.637424 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae488aa6-63da-477b-bd16-0b81111b3566 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:20.639045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ae488aa6-63da-477b-bd16-0b81111b3566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:20.751258 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae488aa6-63da-477b-bd16-0b81111b3566 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:20.764043 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:20.830588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c09ae6-b7fc-4ad0-aafa-6fe7f35cec78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:20.831226 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c09ae6-b7fc-4ad0-aafa-6fe7f35cec78 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:20.831503 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c09ae6-b7fc-4ad0-aafa-6fe7f35cec78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:20.831770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c09ae6-b7fc-4ad0-aafa-6fe7f35cec78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:20.833392 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-36c09ae6-b7fc-4ad0-aafa-6fe7f35cec78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:20.953796 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c09ae6-b7fc-4ad0-aafa-6fe7f35cec78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:20.955250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c09ae6-b7fc-4ad0-aafa-6fe7f35cec78 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:20.985370 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 726/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:23.241115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072e5f2-17cd-4fb2-9f53-7035e6c34581 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:23.241686 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072e5f2-17cd-4fb2-9f53-7035e6c34581 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:23.241686 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072e5f2-17cd-4fb2-9f53-7035e6c34581 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:23.243206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:23.248391 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8574a-c556-436a-9420-0c3562afc408 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:23.249007 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8574a-c556-436a-9420-0c3562afc408 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:23.249155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8574a-c556-436a-9420-0c3562afc408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:23.300480 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:23.320877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-97e8574a-c556-436a-9420-0c3562afc408 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:23.322163 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-97e8574a-c556-436a-9420-0c3562afc408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:23.423559 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:23.708296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1072e5f2-17cd-4fb2-9f53-7035e6c34581 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:23.709868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1072e5f2-17cd-4fb2-9f53-7035e6c34581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:23.822692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1072e5f2-17cd-4fb2-9f53-7035e6c34581 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:23.867432 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:23.903085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09555d-db8e-4db5-a2dc-5d531ee6ac2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:23.903698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09555d-db8e-4db5-a2dc-5d531ee6ac2a service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:23.903976 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09555d-db8e-4db5-a2dc-5d531ee6ac2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:23.904246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09555d-db8e-4db5-a2dc-5d531ee6ac2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:23.906608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ff09555d-db8e-4db5-a2dc-5d531ee6ac2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:24.017094 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09555d-db8e-4db5-a2dc-5d531ee6ac2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:24.018542 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09555d-db8e-4db5-a2dc-5d531ee6ac2a service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:24.037738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 722/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:26.236086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3004cd-8cb2-423b-987b-b6e17a324c0d None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:26.236662 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3004cd-8cb2-423b-987b-b6e17a324c0d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:26.236662 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3004cd-8cb2-423b-987b-b6e17a324c0d None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:26.238000 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:26.243879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e70656-3c72-4463-8492-c3fb876f0349 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:26.244385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e70656-3c72-4463-8492-c3fb876f0349 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:26.244491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e70656-3c72-4463-8492-c3fb876f0349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:26.321226 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-c6e70656-3c72-4463-8492-c3fb876f0349 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:26.322458 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c6e70656-3c72-4463-8492-c3fb876f0349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:26.351909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:26.425360 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:26.711779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3004cd-8cb2-423b-987b-b6e17a324c0d None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:26.713360 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0d3004cd-8cb2-423b-987b-b6e17a324c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:26.829914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3004cd-8cb2-423b-987b-b6e17a324c0d None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:26.851893 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:26.908959 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f639e-aaca-464b-8ddc-ce9994cb67aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:26.909564 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f639e-aaca-464b-8ddc-ce9994cb67aa service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:26.909844 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f639e-aaca-464b-8ddc-ce9994cb67aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:26.910115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f639e-aaca-464b-8ddc-ce9994cb67aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:26.911794 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6b3f639e-aaca-464b-8ddc-ce9994cb67aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:27.024709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f639e-aaca-464b-8ddc-ce9994cb67aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:27.026146 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f639e-aaca-464b-8ddc-ce9994cb67aa service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:27.044968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 729/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:08:29.054467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71280b8d-2abd-4740-b63e-20045293cf97 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:29.055010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71280b8d-2abd-4740-b63e-20045293cf97 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:29.055010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71280b8d-2abd-4740-b63e-20045293cf97 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:29.056287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:29.062465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4c0b6-42a7-47f7-81d9-7032b51530f0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:29.062922 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4c0b6-42a7-47f7-81d9-7032b51530f0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:29.063077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4c0b6-42a7-47f7-81d9-7032b51530f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:29.129001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:29.130588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6fd4c0b6-42a7-47f7-81d9-7032b51530f0 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:29.132343 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6fd4c0b6-42a7-47f7-81d9-7032b51530f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:29.231526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:29.513161 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71280b8d-2abd-4740-b63e-20045293cf97 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:29.515391 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-71280b8d-2abd-4740-b63e-20045293cf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:29.633434 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71280b8d-2abd-4740-b63e-20045293cf97 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:29.684071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:29.712257 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45876b-9c28-44d3-8496-06e012d00703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:29.712866 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45876b-9c28-44d3-8496-06e012d00703 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:29.713143 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45876b-9c28-44d3-8496-06e012d00703 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:29.713418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45876b-9c28-44d3-8496-06e012d00703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:29.715051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fa45876b-9c28-44d3-8496-06e012d00703 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:29.833021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45876b-9c28-44d3-8496-06e012d00703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:29.834474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45876b-9c28-44d3-8496-06e012d00703 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:29.853196 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 725/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:31.872537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa47113-95f5-404e-b31a-bb4be71862e3 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:31.873069 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa47113-95f5-404e-b31a-bb4be71862e3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:31.873069 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa47113-95f5-404e-b31a-bb4be71862e3 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:31.875041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:08:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:31.880657 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3b444-d90c-4327-917c-efe36998fc01 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:31.881097 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3b444-d90c-4327-917c-efe36998fc01 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:31.881210 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3b444-d90c-4327-917c-efe36998fc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:31.952221 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-85b3b444-d90c-4327-917c-efe36998fc01 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:31.953420 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-85b3b444-d90c-4327-917c-efe36998fc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:31.983844 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:32.054107 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:32.339768 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa47113-95f5-404e-b31a-bb4be71862e3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:32.341315 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-dfa47113-95f5-404e-b31a-bb4be71862e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:32.450991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa47113-95f5-404e-b31a-bb4be71862e3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:32.481310 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:32.531421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a37a1e-a452-4418-a21b-1782d259746b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:32.532327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a37a1e-a452-4418-a21b-1782d259746b service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:32.532327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a37a1e-a452-4418-a21b-1782d259746b service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:32.532573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a37a1e-a452-4418-a21b-1782d259746b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:32.534217 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-17a37a1e-a452-4418-a21b-1782d259746b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:32.648488 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a37a1e-a452-4418-a21b-1782d259746b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:32.649932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a37a1e-a452-4418-a21b-1782d259746b service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:32.668840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 732/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:34.678635 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48915c-e62e-4118-8330-a94e659324ef None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:34.679467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48915c-e62e-4118-8330-a94e659324ef None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:34.679467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48915c-e62e-4118-8330-a94e659324ef None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:34.680559 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:34.687157 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e9820-4034-4f52-a804-5833ac88dd7d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:34.687683 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e9820-4034-4f52-a804-5833ac88dd7d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:34.687683 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e9820-4034-4f52-a804-5833ac88dd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:34.758386 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a13e9820-4034-4f52-a804-5833ac88dd7d None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:34.759693 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a13e9820-4034-4f52-a804-5833ac88dd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:34.769565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:34.859912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:08:35.160043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c48915c-e62e-4118-8330-a94e659324ef None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:35.161580 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8c48915c-e62e-4118-8330-a94e659324ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:35.280264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c48915c-e62e-4118-8330-a94e659324ef None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:35.348028 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:35.368623 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94019465-e0ef-46b6-8ac7-18d38f323bac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:35.369228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94019465-e0ef-46b6-8ac7-18d38f323bac service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:35.369506 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94019465-e0ef-46b6-8ac7-18d38f323bac service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:35.369777 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94019465-e0ef-46b6-8ac7-18d38f323bac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:35.371399 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-94019465-e0ef-46b6-8ac7-18d38f323bac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:35.490036 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94019465-e0ef-46b6-8ac7-18d38f323bac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:35.491504 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94019465-e0ef-46b6-8ac7-18d38f323bac service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:35.509941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 728/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:37.581531 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6febdb4-2965-4a1e-872c-af05e6b50542 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:37.582363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6febdb4-2965-4a1e-872c-af05e6b50542 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:37.582552 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6febdb4-2965-4a1e-872c-af05e6b50542 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:37.584110 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:08:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:37.589701 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009fab0-5dc0-47fc-a9b4-84a9fa594a93 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:37.590144 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009fab0-5dc0-47fc-a9b4-84a9fa594a93 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:37.590240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009fab0-5dc0-47fc-a9b4-84a9fa594a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:37.659760 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-3009fab0-5dc0-47fc-a9b4-84a9fa594a93 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:37.660983 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3009fab0-5dc0-47fc-a9b4-84a9fa594a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:37.701733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:37.762506 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:38.051811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6febdb4-2965-4a1e-872c-af05e6b50542 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:38.053368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c6febdb4-2965-4a1e-872c-af05e6b50542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:38.160940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6febdb4-2965-4a1e-872c-af05e6b50542 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:38.199652 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:38.237931 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6361d3-d9f4-4f03-b196-aff172415e62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:38.238532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6361d3-d9f4-4f03-b196-aff172415e62 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:38.238840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6361d3-d9f4-4f03-b196-aff172415e62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:38.239111 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6361d3-d9f4-4f03-b196-aff172415e62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:38.240710 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4a6361d3-d9f4-4f03-b196-aff172415e62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:38.353437 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6361d3-d9f4-4f03-b196-aff172415e62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:38.354909 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6361d3-d9f4-4f03-b196-aff172415e62 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:38.377355 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 735/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:40.390292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b335db7-7deb-4322-a086-9088ced07816 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:40.390989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b335db7-7deb-4322-a086-9088ced07816 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:40.390989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b335db7-7deb-4322-a086-9088ced07816 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:40.392361 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:08:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:40.398099 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6442c3-6de5-4e12-8d04-a1bd5aa156a1 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:40.398588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6442c3-6de5-4e12-8d04-a1bd5aa156a1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:40.398727 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6442c3-6de5-4e12-8d04-a1bd5aa156a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:40.467147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-3b6442c3-6de5-4e12-8d04-a1bd5aa156a1 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:40.468378 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3b6442c3-6de5-4e12-8d04-a1bd5aa156a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:40.487648 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:40.569784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:40.849927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b335db7-7deb-4322-a086-9088ced07816 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:40.851544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9b335db7-7deb-4322-a086-9088ced07816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:40.975701 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b335db7-7deb-4322-a086-9088ced07816 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:41.042643 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:41.050435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7874041-d134-4ea8-9070-6289f7ef17b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:41.051211 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7874041-d134-4ea8-9070-6289f7ef17b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:41.051491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7874041-d134-4ea8-9070-6289f7ef17b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:41.051764 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7874041-d134-4ea8-9070-6289f7ef17b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:41.053417 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f7874041-d134-4ea8-9070-6289f7ef17b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:41.165818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7874041-d134-4ea8-9070-6289f7ef17b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:41.167325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7874041-d134-4ea8-9070-6289f7ef17b2 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:41.190099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 731/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:43.170522 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e33141-87da-44da-81ae-f5069da65f40 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:43.171117 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e33141-87da-44da-81ae-f5069da65f40 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:43.171117 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e33141-87da-44da-81ae-f5069da65f40 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:43.172513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:08:43.177968 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf854c-6019-409e-9d8f-4b2b33e74c34 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:43.178414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf854c-6019-409e-9d8f-4b2b33e74c34 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:43.178550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf854c-6019-409e-9d8f-4b2b33e74c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:43.251618 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f8bf854c-6019-409e-9d8f-4b2b33e74c34 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:08:43.252858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f8bf854c-6019-409e-9d8f-4b2b33e74c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:43.306467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:43.357016 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:08:43.639947 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e33141-87da-44da-81ae-f5069da65f40 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:08:43.641472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-28e33141-87da-44da-81ae-f5069da65f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:43.753340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e33141-87da-44da-81ae-f5069da65f40 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:08:43.802726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:43.831390 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485e133-95f8-4a96-bd64-5ad7d1c3ab54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:08:43.831995 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485e133-95f8-4a96-bd64-5ad7d1c3ab54 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:08:43.832267 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485e133-95f8-4a96-bd64-5ad7d1c3ab54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:08:43.832530 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485e133-95f8-4a96-bd64-5ad7d1c3ab54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:08:43.834118 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8485e133-95f8-4a96-bd64-5ad7d1c3ab54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:08:43.949203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485e133-95f8-4a96-bd64-5ad7d1c3ab54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:08:43.950560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485e133-95f8-4a96-bd64-5ad7d1c3ab54 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:08:43.971782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 738/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:08:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:08:53.355254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=22 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:08:53.969335 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=66 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:03.790100 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53452f7b-d85a-4037-8e0a-22d718552198 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:03.790703 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53452f7b-d85a-4037-8e0a-22d718552198 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:03.790703 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53452f7b-d85a-4037-8e0a-22d718552198 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:03.792066 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 733/1471] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:03.798110 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c8088-3211-4be9-ae65-a94b8d40b31b None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:03.798547 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c8088-3211-4be9-ae65-a94b8d40b31b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:03.798651 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c8088-3211-4be9-ae65-a94b8d40b31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:03.864469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ca0c8088-3211-4be9-ae65-a94b8d40b31b None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:03.865576 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ca0c8088-3211-4be9-ae65-a94b8d40b31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:03.963265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 739/1472] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:09:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:04.264798 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f94e5-208f-4b9c-bb56-f249a881b76a None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:04.265081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f94e5-208f-4b9c-bb56-f249a881b76a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:04.265376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f94e5-208f-4b9c-bb56-f249a881b76a None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:04.266860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 734/1473] 10.222.0.26 () {54 vars in 961 bytes} [Tue Sep 6 16:09:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:04.271912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df0eac-9bfe-46e7-926e-0979e58cf7ae None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:04.272258 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df0eac-9bfe-46e7-926e-0979e58cf7ae None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:04.272581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df0eac-9bfe-46e7-926e-0979e58cf7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:04.355188 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a3df0eac-9bfe-46e7-926e-0979e58cf7ae None None] MFA Rules not processed for user `e50ef7db27744c7ebd2ec95cecc9d1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:04.356423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a3df0eac-9bfe-46e7-926e-0979e58cf7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:04.378489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:04.437606 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 740/1474] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Sep 6 16:09:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:04.445117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f94e5-208f-4b9c-bb56-f249a881b76a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:04.446814 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f56f94e5-208f-4b9c-bb56-f249a881b76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:04.553092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f94e5-208f-4b9c-bb56-f249a881b76a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:04.631327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a421d55-30e8-45dd-b827-065ad9af9bd7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50ef7db27744c7ebd2ec95cecc9d1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:04.631929 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a421d55-30e8-45dd-b827-065ad9af9bd7 service glance] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:04.632202 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a421d55-30e8-45dd-b827-065ad9af9bd7 service glance] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:04.632472 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a421d55-30e8-45dd-b827-065ad9af9bd7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:04.634060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9a421d55-30e8-45dd-b827-065ad9af9bd7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:04.693404 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:04.751964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a421d55-30e8-45dd-b827-065ad9af9bd7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:04.753403 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a421d55-30e8-45dd-b827-065ad9af9bd7 service glance] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:04.772587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 735/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:06.830435 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db365b5e-1d9e-49db-87c9-bfcc87cab3ad None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:06.831023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db365b5e-1d9e-49db-87c9-bfcc87cab3ad None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:06.831023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db365b5e-1d9e-49db-87c9-bfcc87cab3ad None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:06.832416 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 741/1476] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:06.837738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec0a84-c6cf-448f-b68e-898d92f0399e None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:06.838217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec0a84-c6cf-448f-b68e-898d92f0399e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:06.838313 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec0a84-c6cf-448f-b68e-898d92f0399e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:06.906184 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-93ec0a84-c6cf-448f-b68e-898d92f0399e None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:06.907347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-93ec0a84-c6cf-448f-b68e-898d92f0399e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:07.008145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 736/1477] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:07.281106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db365b5e-1d9e-49db-87c9-bfcc87cab3ad None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:07.282638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-db365b5e-1d9e-49db-87c9-bfcc87cab3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:07.409287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db365b5e-1d9e-49db-87c9-bfcc87cab3ad None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:07.463967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:07.522134 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6bf03-b6a8-4a4a-8975-80981d3bfdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:07.522811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6bf03-b6a8-4a4a-8975-80981d3bfdfa admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:07.523092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6bf03-b6a8-4a4a-8975-80981d3bfdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:07.523361 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6bf03-b6a8-4a4a-8975-80981d3bfdfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:07.524034 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6bf03-b6a8-4a4a-8975-80981d3bfdfa admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:07.525777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: WARNING py.warnings [None req-82e6bf03-b6a8-4a4a-8975-80981d3bfdfa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:09:07.525777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: warnings.warn(msg) Sep 06 16:09:07.525777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: Sep 06 16:09:07.527006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6bf03-b6a8-4a4a-8975-80981d3bfdfa admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:07.547312 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 742/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 6 16:09:07 2022] GET /identity/v3/projects => generated 2553 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:09:09.515751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69e281-465c-4186-82bd-53f25c1f6bdf None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:09.516324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69e281-465c-4186-82bd-53f25c1f6bdf None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:09.516324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69e281-465c-4186-82bd-53f25c1f6bdf None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:09.517614 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 737/1479] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:09.522913 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010939c-855b-468d-8190-9a02e3268116 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:09.523372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010939c-855b-468d-8190-9a02e3268116 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:09.523497 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010939c-855b-468d-8190-9a02e3268116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:09.593333 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b010939c-855b-468d-8190-9a02e3268116 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:09.594544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b010939c-855b-468d-8190-9a02e3268116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:09.696241 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 743/1480] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:09.991615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69e281-465c-4186-82bd-53f25c1f6bdf None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:09.993168 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5d69e281-465c-4186-82bd-53f25c1f6bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:10.003699 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:10.151198 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69e281-465c-4186-82bd-53f25c1f6bdf None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:10.234402 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:10.269239 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb590e-66cf-45cd-9267-6721bae5886b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:10.269850 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb590e-66cf-45cd-9267-6721bae5886b service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:10.270131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb590e-66cf-45cd-9267-6721bae5886b service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:10.270400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb590e-66cf-45cd-9267-6721bae5886b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:10.272029 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-90fb590e-66cf-45cd-9267-6721bae5886b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:10.365384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb590e-66cf-45cd-9267-6721bae5886b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:10.367190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb590e-66cf-45cd-9267-6721bae5886b service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:10.385915 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 738/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:12.518328 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0793fe-7db0-4a4c-9e4a-6b1a7376235c None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:12.518988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0793fe-7db0-4a4c-9e4a-6b1a7376235c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:12.518988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0793fe-7db0-4a4c-9e4a-6b1a7376235c None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:12.520270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 744/1482] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:12.525715 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7e16b-157a-4b87-9a8e-02fde69961f2 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:12.526150 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7e16b-157a-4b87-9a8e-02fde69961f2 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:12.526277 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7e16b-157a-4b87-9a8e-02fde69961f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:12.595707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e2b7e16b-157a-4b87-9a8e-02fde69961f2 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:12.596978 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e2b7e16b-157a-4b87-9a8e-02fde69961f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:12.648681 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:12.701304 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 739/1483] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:12.990098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0793fe-7db0-4a4c-9e4a-6b1a7376235c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:12.991754 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0d0793fe-7db0-4a4c-9e4a-6b1a7376235c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:13.017969 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:13.101916 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0793fe-7db0-4a4c-9e4a-6b1a7376235c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:13.177397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bdabd4-d779-4fec-8d91-ce3f12caa3d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:13.177995 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdabd4-d779-4fec-8d91-ce3f12caa3d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:13.178274 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdabd4-d779-4fec-8d91-ce3f12caa3d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:13.178546 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdabd4-d779-4fec-8d91-ce3f12caa3d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:13.180200 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-51bdabd4-d779-4fec-8d91-ce3f12caa3d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:13.259469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:13.297842 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bdabd4-d779-4fec-8d91-ce3f12caa3d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:13.299322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bdabd4-d779-4fec-8d91-ce3f12caa3d5 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:13.318479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 745/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:15.330631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d622d-7ce9-4b1d-99fe-1aaa637ff3f1 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:15.331222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d622d-7ce9-4b1d-99fe-1aaa637ff3f1 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:15.331222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d622d-7ce9-4b1d-99fe-1aaa637ff3f1 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:15.332542 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 740/1485] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:15.338055 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f528d-759f-4ce8-a5b7-4d710c763d91 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:15.338544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f528d-759f-4ce8-a5b7-4d710c763d91 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:15.338654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f528d-759f-4ce8-a5b7-4d710c763d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:15.406935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d76f528d-759f-4ce8-a5b7-4d710c763d91 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:15.408157 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d76f528d-759f-4ce8-a5b7-4d710c763d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:15.509483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 746/1486] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:15.801343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d622d-7ce9-4b1d-99fe-1aaa637ff3f1 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:15.802909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4c3d622d-7ce9-4b1d-99fe-1aaa637ff3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:15.908892 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d622d-7ce9-4b1d-99fe-1aaa637ff3f1 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:15.985169 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:15.987134 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebfa3c-7615-41c1-9286-e41734b7c34a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:15.987905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebfa3c-7615-41c1-9286-e41734b7c34a service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:15.988194 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebfa3c-7615-41c1-9286-e41734b7c34a service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:15.988667 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebfa3c-7615-41c1-9286-e41734b7c34a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:15.990614 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-73ebfa3c-7615-41c1-9286-e41734b7c34a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:16.102930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebfa3c-7615-41c1-9286-e41734b7c34a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:16.104388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebfa3c-7615-41c1-9286-e41734b7c34a service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:16.123353 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 741/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:18.174770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83022a6f-7860-4808-ab3c-b4f88b6f7d83 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:18.175494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83022a6f-7860-4808-ab3c-b4f88b6f7d83 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:18.175494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83022a6f-7860-4808-ab3c-b4f88b6f7d83 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:18.176808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 747/1488] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:18.183751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d011bc-d2dd-457b-a2c1-271fd1c49a8c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:18.184285 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d011bc-d2dd-457b-a2c1-271fd1c49a8c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:18.185304 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d011bc-d2dd-457b-a2c1-271fd1c49a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:18.261220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f9d011bc-d2dd-457b-a2c1-271fd1c49a8c None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:18.262451 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f9d011bc-d2dd-457b-a2c1-271fd1c49a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:18.341969 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:18.399955 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 742/1489] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:18.697267 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83022a6f-7860-4808-ab3c-b4f88b6f7d83 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:18.698873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-83022a6f-7860-4808-ab3c-b4f88b6f7d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:18.739553 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:18.814445 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83022a6f-7860-4808-ab3c-b4f88b6f7d83 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:18.892593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da979c8-5df1-4ba1-81e0-1766c23c9cae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:18.893108 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da979c8-5df1-4ba1-81e0-1766c23c9cae service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:18.893379 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da979c8-5df1-4ba1-81e0-1766c23c9cae service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:18.893642 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da979c8-5df1-4ba1-81e0-1766c23c9cae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:18.895270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9da979c8-5df1-4ba1-81e0-1766c23c9cae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:18.985436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:19.020545 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da979c8-5df1-4ba1-81e0-1766c23c9cae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:19.021992 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da979c8-5df1-4ba1-81e0-1766c23c9cae service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:19.043476 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 748/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:21.242124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216a238-3400-44e1-a6ca-db6f1f66f40d None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:21.242717 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216a238-3400-44e1-a6ca-db6f1f66f40d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:21.242717 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216a238-3400-44e1-a6ca-db6f1f66f40d None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:21.244037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 743/1491] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 6 16:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:09:21.250145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cb209-5266-4f4b-9037-5665908d05cd None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:21.250609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cb209-5266-4f4b-9037-5665908d05cd None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:21.250739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cb209-5266-4f4b-9037-5665908d05cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:21.324064 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ba0cb209-5266-4f4b-9037-5665908d05cd None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:09:21.326113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ba0cb209-5266-4f4b-9037-5665908d05cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:21.430948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 749/1492] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 6 16:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:09:21.723722 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216a238-3400-44e1-a6ca-db6f1f66f40d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:09:21.725286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6216a238-3400-44e1-a6ca-db6f1f66f40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:21.834893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216a238-3400-44e1-a6ca-db6f1f66f40d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:09:21.914026 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad54f1d-136b-4b2c-a965-c8616701b855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:09:21.914544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad54f1d-136b-4b2c-a965-c8616701b855 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:09:21.914856 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad54f1d-136b-4b2c-a965-c8616701b855 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:09:21.915129 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad54f1d-136b-4b2c-a965-c8616701b855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:09:21.916748 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4ad54f1d-136b-4b2c-a965-c8616701b855 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:09:21.926730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:22.031865 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad54f1d-136b-4b2c-a965-c8616701b855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:09:22.033308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad54f1d-136b-4b2c-a965-c8616701b855 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:09:22.052482 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 744/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:09:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:09:31.429009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=82 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:09:32.050197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=50 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:38.569312 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fa0e7-e736-4e69-b41c-2751447fe300 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:38.569938 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fa0e7-e736-4e69-b41c-2751447fe300 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:38.569938 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fa0e7-e736-4e69-b41c-2751447fe300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:38.650748 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e75fa0e7-e736-4e69-b41c-2751447fe300 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:38.652471 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e75fa0e7-e736-4e69-b41c-2751447fe300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:38.752745 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 750/1494] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:38.761186 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7b72d-5d37-482e-8a87-ea101acccf89 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:38.762358 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7b72d-5d37-482e-8a87-ea101acccf89 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:38.762358 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7b72d-5d37-482e-8a87-ea101acccf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:38.843476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-27f7b72d-5d37-482e-8a87-ea101acccf89 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:38.844710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-27f7b72d-5d37-482e-8a87-ea101acccf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:38.945714 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 745/1495] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:38.951227 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fa0e7-e736-4e69-b41c-2751447fe300 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:38.952453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e75fa0e7-e736-4e69-b41c-2751447fe300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:38.964150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fa0e7-e736-4e69-b41c-2751447fe300 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:39.029870 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:39.065572 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72503b4b-61c8-4e6a-8949-9c849039ae94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:39.066179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72503b4b-61c8-4e6a-8949-9c849039ae94 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:39.066457 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72503b4b-61c8-4e6a-8949-9c849039ae94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:39.066764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72503b4b-61c8-4e6a-8949-9c849039ae94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:39.067409 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72503b4b-61c8-4e6a-8949-9c849039ae94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:10:39.067703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72503b4b-61c8-4e6a-8949-9c849039ae94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:39.068718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: WARNING py.warnings [None req-72503b4b-61c8-4e6a-8949-9c849039ae94 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:10:39.068718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: warnings.warn(msg) Sep 06 16:10:39.068718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: Sep 06 16:10:39.069603 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72503b4b-61c8-4e6a-8949-9c849039ae94 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:39.082245 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 751/1496] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:10:39.091363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7b72d-5d37-482e-8a87-ea101acccf89 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:39.100735 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7b72d-5d37-482e-8a87-ea101acccf89 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:39.166092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:39.205026 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e811959-035b-4aad-96d1-f4448e5f9f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:39.205619 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e811959-035b-4aad-96d1-f4448e5f9f4c admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:39.205899 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e811959-035b-4aad-96d1-f4448e5f9f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:39.206163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e811959-035b-4aad-96d1-f4448e5f9f4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:39.206820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e811959-035b-4aad-96d1-f4448e5f9f4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:39.208602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: WARNING py.warnings [None req-1e811959-035b-4aad-96d1-f4448e5f9f4c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:10:39.208602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: warnings.warn(msg) Sep 06 16:10:39.208602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: Sep 06 16:10:39.209807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e811959-035b-4aad-96d1-f4448e5f9f4c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:39.271814 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 746/1497] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:10:39 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:10:39.276358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72503b4b-61c8-4e6a-8949-9c849039ae94 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:39.277915 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-72503b4b-61c8-4e6a-8949-9c849039ae94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:39.314544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72503b4b-61c8-4e6a-8949-9c849039ae94 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:39.418081 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b164f7-34c9-4c68-92fe-0665e36149b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:39.418589 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b164f7-34c9-4c68-92fe-0665e36149b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:39.418897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b164f7-34c9-4c68-92fe-0665e36149b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:39.419295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b164f7-34c9-4c68-92fe-0665e36149b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:39.419801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b164f7-34c9-4c68-92fe-0665e36149b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:39.420824 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: WARNING py.warnings [None req-84b164f7-34c9-4c68-92fe-0665e36149b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:10:39.420824 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: warnings.warn(msg) Sep 06 16:10:39.420824 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: Sep 06 16:10:39.422051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b164f7-34c9-4c68-92fe-0665e36149b2 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:39.466952 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 752/1498] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:10:39 2022] POST /identity/v3/users => generated 289 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:10:39.471359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e811959-035b-4aad-96d1-f4448e5f9f4c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:39.472944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1e811959-035b-4aad-96d1-f4448e5f9f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:39.537130 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e811959-035b-4aad-96d1-f4448e5f9f4c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:39.625201 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:39.636694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cf68a-fabf-4a74-88cb-171ff6178bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:39.637307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cf68a-fabf-4a74-88cb-171ff6178bab admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:39.637584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cf68a-fabf-4a74-88cb-171ff6178bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:39.637860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cf68a-fabf-4a74-88cb-171ff6178bab admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:39.638530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994cf68a-fabf-4a74-88cb-171ff6178bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:39.639588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: WARNING py.warnings [None req-994cf68a-fabf-4a74-88cb-171ff6178bab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:10:39.639588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: warnings.warn(msg) Sep 06 16:10:39.639588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: Sep 06 16:10:39.640479 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994cf68a-fabf-4a74-88cb-171ff6178bab admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:39.653799 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 747/1499] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:10:39.658408 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b164f7-34c9-4c68-92fe-0665e36149b2 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:39.659967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-84b164f7-34c9-4c68-92fe-0665e36149b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:39.672748 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b164f7-34c9-4c68-92fe-0665e36149b2 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:39.699680 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:39.774290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37b130-30ac-4c4a-aed6-83cd49b0c0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:39.774931 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37b130-30ac-4c4a-aed6-83cd49b0c0ec admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:39.775208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37b130-30ac-4c4a-aed6-83cd49b0c0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:39.775476 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37b130-30ac-4c4a-aed6-83cd49b0c0ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:39.776120 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37b130-30ac-4c4a-aed6-83cd49b0c0ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:39.778266 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: WARNING py.warnings [None req-ed37b130-30ac-4c4a-aed6-83cd49b0c0ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:10:39.778266 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: warnings.warn(msg) Sep 06 16:10:39.778266 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: Sep 06 16:10:39.779186 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37b130-30ac-4c4a-aed6-83cd49b0c0ec admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:39.792376 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 753/1500] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:10:39.796499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cf68a-fabf-4a74-88cb-171ff6178bab admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:39.798279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-994cf68a-fabf-4a74-88cb-171ff6178bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:39.834070 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cf68a-fabf-4a74-88cb-171ff6178bab admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:39.943715 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ca06c-51e1-4d21-ba32-724000e143e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:39.944481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ca06c-51e1-4d21-ba32-724000e143e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:39.944808 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ca06c-51e1-4d21-ba32-724000e143e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:39.945072 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ca06c-51e1-4d21-ba32-724000e143e3 admin admin] PATH_INFO: `/v3/projects/ef3ee01cfa324353a8174a316652d745/users/b60d9b4cb61b4648bc719c7d8a4788ed/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:39.971915 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ca06c-51e1-4d21-ba32-724000e143e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef3ee01cfa324353a8174a316652d745, user_id=b60d9b4cb61b4648bc719c7d8a4788ed, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:39.972964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: WARNING py.warnings [None req-9e4ca06c-51e1-4d21-ba32-724000e143e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:10:39.972964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: warnings.warn(msg) Sep 06 16:10:39.972964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: Sep 06 16:10:39.974975 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ca06c-51e1-4d21-ba32-724000e143e3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:40.001315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 748/1501] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:10:39 2022] PUT /identity/v3/projects/ef3ee01cfa324353a8174a316652d745/users/b60d9b4cb61b4648bc719c7d8a4788ed/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:10:40.014157 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4df6ea-641a-4415-ba9d-986134fb7bd3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:40.014441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4df6ea-641a-4415-ba9d-986134fb7bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:40.014756 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4df6ea-641a-4415-ba9d-986134fb7bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:40.077509 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-cc4df6ea-641a-4415-ba9d-986134fb7bd3 None None] MFA Rules not processed for user `b60d9b4cb61b4648bc719c7d8a4788ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:40.078772 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cc4df6ea-641a-4415-ba9d-986134fb7bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:40.166097 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 754/1502] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:10:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:40.179652 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee43f6-5ce8-4314-88ae-017b0a2329b9 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:40.179938 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee43f6-5ce8-4314-88ae-017b0a2329b9 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:40.180215 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee43f6-5ce8-4314-88ae-017b0a2329b9 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:40.181554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 749/1503] 10.222.0.26 () {54 vars in 964 bytes} [Tue Sep 6 16:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:10:40.187671 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961a2f5-3b15-415a-828a-5c7144e75d49 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:40.187928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961a2f5-3b15-415a-828a-5c7144e75d49 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:40.188209 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961a2f5-3b15-415a-828a-5c7144e75d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:40.224418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:40.273255 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4961a2f5-3b15-415a-828a-5c7144e75d49 None None] MFA Rules not processed for user `7849ef049ffd4495909f997735af8782`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:40.274040 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4961a2f5-3b15-415a-828a-5c7144e75d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:40.353488 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 755/1504] 10.222.0.26 () {58 vars in 1061 bytes} [Tue Sep 6 16:10:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:40.363443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee43f6-5ce8-4314-88ae-017b0a2329b9 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:40.364969 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c1ee43f6-5ce8-4314-88ae-017b0a2329b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:40.421480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:40.481140 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee43f6-5ce8-4314-88ae-017b0a2329b9 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:40.561544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdfd1a-6ff9-4d44-9f8b-f4a499c05704 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:40.562154 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdfd1a-6ff9-4d44-9f8b-f4a499c05704 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:40.562428 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdfd1a-6ff9-4d44-9f8b-f4a499c05704 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:40.562720 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdfd1a-6ff9-4d44-9f8b-f4a499c05704 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:40.564358 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-25bdfd1a-6ff9-4d44-9f8b-f4a499c05704 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:40.668538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bdfd1a-6ff9-4d44-9f8b-f4a499c05704 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:40.669980 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bdfd1a-6ff9-4d44-9f8b-f4a499c05704 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:40.673628 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:40.690205 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 750/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:10:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:10:40.741754 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577db7c-3dd8-4d4b-ae02-e7e42765937b None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:40.742266 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577db7c-3dd8-4d4b-ae02-e7e42765937b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:40.742371 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577db7c-3dd8-4d4b-ae02-e7e42765937b None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:40.743671 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 756/1506] 10.222.0.26 () {52 vars in 830 bytes} [Tue Sep 6 16:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:10:40.748102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdfd1a-6ff9-4d44-9f8b-f4a499c05704 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:40.749370 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-25bdfd1a-6ff9-4d44-9f8b-f4a499c05704 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:40.852630 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdfd1a-6ff9-4d44-9f8b-f4a499c05704 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:40.949161 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faf479-6e3a-4244-a564-bccd525a0a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:40.949789 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faf479-6e3a-4244-a564-bccd525a0a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:40.950064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faf479-6e3a-4244-a564-bccd525a0a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:40.950332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faf479-6e3a-4244-a564-bccd525a0a5d admin admin] PATH_INFO: `/v3/users/b60d9b4cb61b4648bc719c7d8a4788ed` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:40.963504 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28faf479-6e3a-4244-a564-bccd525a0a5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b60d9b4cb61b4648bc719c7d8a4788ed)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:40.965086 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: WARNING py.warnings [None req-28faf479-6e3a-4244-a564-bccd525a0a5d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:10:40.965086 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: warnings.warn(msg) Sep 06 16:10:40.965086 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: Sep 06 16:10:40.966320 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28faf479-6e3a-4244-a564-bccd525a0a5d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:40.976556 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:41.037160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-28faf479-6e3a-4244-a564-bccd525a0a5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b60d9b4cb61b4648bc719c7d8a4788ed'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:10:41.053074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-28faf479-6e3a-4244-a564-bccd525a0a5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b60d9b4cb61b4648bc719c7d8a4788ed'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:10:41.078412 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-28faf479-6e3a-4244-a564-bccd525a0a5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b60d9b4cb61b4648bc719c7d8a4788ed'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:10:41.078965 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-28faf479-6e3a-4244-a564-bccd525a0a5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b60d9b4cb61b4648bc719c7d8a4788ed'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:10:41.079476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-28faf479-6e3a-4244-a564-bccd525a0a5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b60d9b4cb61b4648bc719c7d8a4788ed'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:10:41.096507 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 751/1507] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:10:40 2022] DELETE /identity/v3/users/b60d9b4cb61b4648bc719c7d8a4788ed => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:10:41.101126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323846d-a7f5-47b6-8328-d0b2029c7091 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:41.101416 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323846d-a7f5-47b6-8328-d0b2029c7091 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:41.101697 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323846d-a7f5-47b6-8328-d0b2029c7091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:41.163580 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6323846d-a7f5-47b6-8328-d0b2029c7091 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:10:41.164864 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6323846d-a7f5-47b6-8328-d0b2029c7091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:41.245557 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:41.259383 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 757/1508] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:10:41.269816 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faf479-6e3a-4244-a564-bccd525a0a5d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:41.271088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-28faf479-6e3a-4244-a564-bccd525a0a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:41.379246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faf479-6e3a-4244-a564-bccd525a0a5d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:41.562489 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:41.585819 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d825571-c990-478a-8848-96f2b1bcad84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:41.586427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d825571-c990-478a-8848-96f2b1bcad84 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:41.586736 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d825571-c990-478a-8848-96f2b1bcad84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:41.587007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d825571-c990-478a-8848-96f2b1bcad84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:41.588622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6d825571-c990-478a-8848-96f2b1bcad84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:41.714402 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d825571-c990-478a-8848-96f2b1bcad84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:41.716007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d825571-c990-478a-8848-96f2b1bcad84 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:41.736156 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 752/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:10:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:10:42.137527 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323846d-a7f5-47b6-8328-d0b2029c7091 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:10:42.139433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6323846d-a7f5-47b6-8328-d0b2029c7091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:10:42.154282 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323846d-a7f5-47b6-8328-d0b2029c7091 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:10:42.265706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0558518-3915-485c-b7f0-ef745cecf2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:10:42.266253 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0558518-3915-485c-b7f0-ef745cecf2f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:10:42.266531 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0558518-3915-485c-b7f0-ef745cecf2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:10:42.266826 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0558518-3915-485c-b7f0-ef745cecf2f0 admin admin] PATH_INFO: `/v3/projects/ef3ee01cfa324353a8174a316652d745` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:10:42.279058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0558518-3915-485c-b7f0-ef745cecf2f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef3ee01cfa324353a8174a316652d745)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:10:42.280187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: WARNING py.warnings [None req-d0558518-3915-485c-b7f0-ef745cecf2f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:10:42.280187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: warnings.warn(msg) Sep 06 16:10:42.280187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: Sep 06 16:10:42.281425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0558518-3915-485c-b7f0-ef745cecf2f0 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:10:42.345920 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d0558518-3915-485c-b7f0-ef745cecf2f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef3ee01cfa324353a8174a316652d745', 'request_id': 'req-d0558518-3915-485c-b7f0-ef745cecf2f0'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:10:42.361861 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d0558518-3915-485c-b7f0-ef745cecf2f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef3ee01cfa324353a8174a316652d745', 'request_id': 'req-d0558518-3915-485c-b7f0-ef745cecf2f0'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:10:42.368963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d0558518-3915-485c-b7f0-ef745cecf2f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef3ee01cfa324353a8174a316652d745', 'request_id': 'req-d0558518-3915-485c-b7f0-ef745cecf2f0'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:10:42.399085 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d0558518-3915-485c-b7f0-ef745cecf2f0 admin admin] The token cache is being invalidate because project ef3ee01cfa324353a8174a316652d745 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:10:42.399358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d0558518-3915-485c-b7f0-ef745cecf2f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:10:42.399900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d0558518-3915-485c-b7f0-ef745cecf2f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:10:42.401236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 758/1510] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:10:42 2022] DELETE /identity/v3/projects/ef3ee01cfa324353a8174a316652d745 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:10:51.733481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=62 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:10:52.396539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:12:10.957278 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eff8c9-f4c4-44d4-8fb6-069927ce7043 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:10.957846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eff8c9-f4c4-44d4-8fb6-069927ce7043 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:10.957846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eff8c9-f4c4-44d4-8fb6-069927ce7043 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:10.959128 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:12:10.964025 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c951f93-0929-4333-bfa2-92b3af0a3efa None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:10.964506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c951f93-0929-4333-bfa2-92b3af0a3efa None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:10.964644 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c951f93-0929-4333-bfa2-92b3af0a3efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:11.048027 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8c951f93-0929-4333-bfa2-92b3af0a3efa None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:12:11.049226 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8c951f93-0929-4333-bfa2-92b3af0a3efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:11.153986 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:12:11.309675 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eff8c9-f4c4-44d4-8fb6-069927ce7043 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:11.311941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-99eff8c9-f4c4-44d4-8fb6-069927ce7043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:11.433690 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eff8c9-f4c4-44d4-8fb6-069927ce7043 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:11.440767 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19912ed4-08c7-414e-b331-a832c49e6384 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:11.441488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19912ed4-08c7-414e-b331-a832c49e6384 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:11.441770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19912ed4-08c7-414e-b331-a832c49e6384 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:11.442039 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19912ed4-08c7-414e-b331-a832c49e6384 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:11.450265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19912ed4-08c7-414e-b331-a832c49e6384 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:11.451914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19912ed4-08c7-414e-b331-a832c49e6384 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:11.462436 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 754/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:12:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:12:12.298892 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd94bb8-7864-4259-8ba5-bb41165174bb None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:12.299479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd94bb8-7864-4259-8ba5-bb41165174bb None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:12.299479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd94bb8-7864-4259-8ba5-bb41165174bb None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:12.300920 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:12:12.305677 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add437b9-bf26-4a2c-a6ec-776413c237c0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:12.306080 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add437b9-bf26-4a2c-a6ec-776413c237c0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:12.306228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add437b9-bf26-4a2c-a6ec-776413c237c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:12.326574 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-add437b9-bf26-4a2c-a6ec-776413c237c0 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:12:12.328394 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-add437b9-bf26-4a2c-a6ec-776413c237c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:12.336393 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:12:12.488941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd94bb8-7864-4259-8ba5-bb41165174bb None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:12.503464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd94bb8-7864-4259-8ba5-bb41165174bb None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:12.508872 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc53f5-1a47-4674-8790-ed47fb142f6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:12.509484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc53f5-1a47-4674-8790-ed47fb142f6e service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:12.509756 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc53f5-1a47-4674-8790-ed47fb142f6e service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:12.510027 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc53f5-1a47-4674-8790-ed47fb142f6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:12.519573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc53f5-1a47-4674-8790-ed47fb142f6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:12.522109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc53f5-1a47-4674-8790-ed47fb142f6e service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:12.534492 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 761/1516] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:12:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:12:13.107913 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3338a9f-08f8-46df-90c1-a926fa2fba81 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:13.108178 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3338a9f-08f8-46df-90c1-a926fa2fba81 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:13.108475 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3338a9f-08f8-46df-90c1-a926fa2fba81 None None] PATH_INFO: `/v3` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:13.109752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 756/1517] 10.222.0.20 () {54 vars in 925 bytes} [Tue Sep 6 16:12:13 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:12:13.115826 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5088a-e195-439d-b7e7-b11fdb97d630 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:13.116145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5088a-e195-439d-b7e7-b11fdb97d630 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:13.116421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5088a-e195-439d-b7e7-b11fdb97d630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:13.179196 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-44a5088a-e195-439d-b7e7-b11fdb97d630 None None] MFA Rules not processed for user `034d75ab747e4c0cab040e47c6a6236e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:12:13.180416 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-44a5088a-e195-439d-b7e7-b11fdb97d630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:13.198314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:12:13.254560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 762/1518] 10.222.0.20 () {58 vars in 1015 bytes} [Tue Sep 6 16:12:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:12:22.321168 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=56 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:12:22.381651 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4982c2-8043-4d4c-ac5c-73b4d00c3dbe None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:22.381852 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4982c2-8043-4d4c-ac5c-73b4d00c3dbe None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:22.382131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4982c2-8043-4d4c-ac5c-73b4d00c3dbe None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:22.383505 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 757/1519] 10.222.0.26 () {54 vars in 961 bytes} [Tue Sep 6 16:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:12:22.388573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb82c6-0920-4a7e-813d-b4cbd7af45c2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:22.389066 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb82c6-0920-4a7e-813d-b4cbd7af45c2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:22.389176 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb82c6-0920-4a7e-813d-b4cbd7af45c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:22.409863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-11bb82c6-0920-4a7e-813d-b4cbd7af45c2 None None] MFA Rules not processed for user `034d75ab747e4c0cab040e47c6a6236e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:12:22.411102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-11bb82c6-0920-4a7e-813d-b4cbd7af45c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:22.419572 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 763/1520] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Sep 6 16:12:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:12:22.428050 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4982c2-8043-4d4c-ac5c-73b4d00c3dbe None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:22.442622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4982c2-8043-4d4c-ac5c-73b4d00c3dbe None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:22.447590 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068e792-e73c-4bb3-8c8c-f7c8c6170bc1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034d75ab747e4c0cab040e47c6a6236e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:22.448209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068e792-e73c-4bb3-8c8c-f7c8c6170bc1 service placement] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:22.448491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068e792-e73c-4bb3-8c8c-f7c8c6170bc1 service placement] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:22.448766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068e792-e73c-4bb3-8c8c-f7c8c6170bc1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:22.457155 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068e792-e73c-4bb3-8c8c-f7c8c6170bc1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:22.458756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068e792-e73c-4bb3-8c8c-f7c8c6170bc1 service placement] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:22.468425 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 758/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:12:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:12:23.769662 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7c1df-84e7-41d7-9d62-1466f1d1ec33 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:23.770296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7c1df-84e7-41d7-9d62-1466f1d1ec33 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:23.770296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7c1df-84e7-41d7-9d62-1466f1d1ec33 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:23.771593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:12:23.776569 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b63d52-26d2-4f83-907e-b0682990e534 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:23.777011 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b63d52-26d2-4f83-907e-b0682990e534 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:23.777127 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b63d52-26d2-4f83-907e-b0682990e534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:23.797957 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-76b63d52-26d2-4f83-907e-b0682990e534 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:12:23.799197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-76b63d52-26d2-4f83-907e-b0682990e534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:23.807025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:12:23.960171 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7c1df-84e7-41d7-9d62-1466f1d1ec33 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:23.975135 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7c1df-84e7-41d7-9d62-1466f1d1ec33 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:23.980558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b5e33-8aaa-4274-8ed5-b3f4fab4c1fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:23.981158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b5e33-8aaa-4274-8ed5-b3f4fab4c1fb service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:23.981437 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b5e33-8aaa-4274-8ed5-b3f4fab4c1fb service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:23.981705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b5e33-8aaa-4274-8ed5-b3f4fab4c1fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:23.990683 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b5e33-8aaa-4274-8ed5-b3f4fab4c1fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:23.992260 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b5e33-8aaa-4274-8ed5-b3f4fab4c1fb service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:24.002148 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 765/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:12:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:12:24.824430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9ebb4-0b7c-4a90-9b71-5377d5eea314 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:24.824959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9ebb4-0b7c-4a90-9b71-5377d5eea314 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:24.824959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9ebb4-0b7c-4a90-9b71-5377d5eea314 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:24.826332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 16:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:12:24.831930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0c14d-5ef5-4d17-9133-b3b7ef2716d3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:24.832387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0c14d-5ef5-4d17-9133-b3b7ef2716d3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:24.832494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0c14d-5ef5-4d17-9133-b3b7ef2716d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:24.852366 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d8d0c14d-5ef5-4d17-9133-b3b7ef2716d3 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:12:24.853469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d8d0c14d-5ef5-4d17-9133-b3b7ef2716d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:24.861346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 16:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:12:25.013057 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9ebb4-0b7c-4a90-9b71-5377d5eea314 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:25.027736 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9ebb4-0b7c-4a90-9b71-5377d5eea314 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:25.034267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8c63d-0bfb-489c-b1e0-806fb77d183c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:25.034894 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8c63d-0bfb-489c-b1e0-806fb77d183c service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:25.035172 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8c63d-0bfb-489c-b1e0-806fb77d183c service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:25.035444 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8c63d-0bfb-489c-b1e0-806fb77d183c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:25.043635 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8c63d-0bfb-489c-b1e0-806fb77d183c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:25.045228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8c63d-0bfb-489c-b1e0-806fb77d183c service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:25.055659 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 761/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:12:29.040589 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04a432-acdd-4148-a4cc-4dc428ffe46a None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:29.041112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04a432-acdd-4148-a4cc-4dc428ffe46a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:29.041112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04a432-acdd-4148-a4cc-4dc428ffe46a None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:29.042409 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:12:29.048389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd65e6-a96e-4014-8627-7112ec30c850 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:29.048878 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd65e6-a96e-4014-8627-7112ec30c850 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:29.048993 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd65e6-a96e-4014-8627-7112ec30c850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:29.069794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e4cd65e6-a96e-4014-8627-7112ec30c850 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:12:29.071059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e4cd65e6-a96e-4014-8627-7112ec30c850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:29.079350 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:12:29.369713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04a432-acdd-4148-a4cc-4dc428ffe46a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:29.371455 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ec04a432-acdd-4148-a4cc-4dc428ffe46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:29.453380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04a432-acdd-4148-a4cc-4dc428ffe46a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:29.457077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347621be-8252-4831-990c-13aa25b12af6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:29.457682 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347621be-8252-4831-990c-13aa25b12af6 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:29.457963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347621be-8252-4831-990c-13aa25b12af6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:29.458292 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347621be-8252-4831-990c-13aa25b12af6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:29.467670 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347621be-8252-4831-990c-13aa25b12af6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:29.469366 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347621be-8252-4831-990c-13aa25b12af6 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:29.478855 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 768/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:12:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:12:31.486699 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9093ce-c0e4-4f27-bb80-a001a091b984 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:31.487314 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9093ce-c0e4-4f27-bb80-a001a091b984 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:31.487314 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9093ce-c0e4-4f27-bb80-a001a091b984 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:31.488608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:12:31.493867 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646bbde-a3f1-4b17-8e91-633b4a778e91 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:31.494278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646bbde-a3f1-4b17-8e91-633b4a778e91 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:31.494393 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646bbde-a3f1-4b17-8e91-633b4a778e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:31.521110 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-3646bbde-a3f1-4b17-8e91-633b4a778e91 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:12:31.522310 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3646bbde-a3f1-4b17-8e91-633b4a778e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:31.530086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:12:31.813482 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9093ce-c0e4-4f27-bb80-a001a091b984 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:31.824174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9093ce-c0e4-4f27-bb80-a001a091b984 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:31.827773 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6cd48-6f95-49d3-9cbd-df1d9b498434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:31.828383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6cd48-6f95-49d3-9cbd-df1d9b498434 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:31.828664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6cd48-6f95-49d3-9cbd-df1d9b498434 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:31.828939 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6cd48-6f95-49d3-9cbd-df1d9b498434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:31.837753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b6cd48-6f95-49d3-9cbd-df1d9b498434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:31.839255 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b6cd48-6f95-49d3-9cbd-df1d9b498434 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:31.849801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 764/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:12:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:12:35.405352 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cc0b0-e48b-4ec5-a81b-6aa4d1e178e8 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:35.406975 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cc0b0-e48b-4ec5-a81b-6aa4d1e178e8 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:35.406975 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cc0b0-e48b-4ec5-a81b-6aa4d1e178e8 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:35.407923 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:12:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:12:35.413956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ec74f-9eb6-4f4c-948e-ee18cf2d0317 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:35.414397 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ec74f-9eb6-4f4c-948e-ee18cf2d0317 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:35.414527 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ec74f-9eb6-4f4c-948e-ee18cf2d0317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:35.435332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-5d8ec74f-9eb6-4f4c-948e-ee18cf2d0317 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:12:35.436527 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5d8ec74f-9eb6-4f4c-948e-ee18cf2d0317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:35.444592 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:12:36.015534 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cc0b0-e48b-4ec5-a81b-6aa4d1e178e8 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:36.017166 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ce9cc0b0-e48b-4ec5-a81b-6aa4d1e178e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:36.083757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:12:36.097668 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cc0b0-e48b-4ec5-a81b-6aa4d1e178e8 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:36.101338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050feea7-b8b4-438e-a210-01a1997bb63c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:36.101939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050feea7-b8b4-438e-a210-01a1997bb63c service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:36.102221 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050feea7-b8b4-438e-a210-01a1997bb63c service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:36.102496 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050feea7-b8b4-438e-a210-01a1997bb63c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:36.112120 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050feea7-b8b4-438e-a210-01a1997bb63c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:36.113545 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050feea7-b8b4-438e-a210-01a1997bb63c service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:36.122898 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 771/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:12:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:12:38.106340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8251caf-69c6-4960-9ca5-7e6921663c32 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:38.106871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8251caf-69c6-4960-9ca5-7e6921663c32 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:38.106871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8251caf-69c6-4960-9ca5-7e6921663c32 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:38.108185 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 16:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:12:38.113859 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556e50e-f6b3-4b37-a509-39c2a275d5b0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:38.114278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556e50e-f6b3-4b37-a509-39c2a275d5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:38.114413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556e50e-f6b3-4b37-a509-39c2a275d5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:38.134251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-9556e50e-f6b3-4b37-a509-39c2a275d5b0 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:12:38.135466 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9556e50e-f6b3-4b37-a509-39c2a275d5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:38.144764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 16:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:12:38.713344 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8251caf-69c6-4960-9ca5-7e6921663c32 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:12:38.715153 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a8251caf-69c6-4960-9ca5-7e6921663c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:12:38.728673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8251caf-69c6-4960-9ca5-7e6921663c32 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:12:38.732423 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f30cb1-87e7-4a45-a769-8d239ed691db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:12:38.733098 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f30cb1-87e7-4a45-a769-8d239ed691db service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:12:38.733381 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f30cb1-87e7-4a45-a769-8d239ed691db service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:12:38.733694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f30cb1-87e7-4a45-a769-8d239ed691db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:12:38.743467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f30cb1-87e7-4a45-a769-8d239ed691db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:12:38.744894 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f30cb1-87e7-4a45-a769-8d239ed691db service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:12:38.755020 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 767/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:12:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:12:48.129560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=11 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:12:48.750944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=33 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:06.201789 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e967914-861e-48d1-9383-c5f963a0b900 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.202432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e967914-861e-48d1-9383-c5f963a0b900 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.202432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e967914-861e-48d1-9383-c5f963a0b900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.235785 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8e967914-861e-48d1-9383-c5f963a0b900 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:06.237065 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8e967914-861e-48d1-9383-c5f963a0b900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:06.250579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 773/1540] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:06.259801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5db2c2-5fd2-4315-9da0-1e54a4d402c1 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.260424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5db2c2-5fd2-4315-9da0-1e54a4d402c1 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.260424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5db2c2-5fd2-4315-9da0-1e54a4d402c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.269043 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6742ce-fc78-4fe0-9e75-1baa40ee03f9 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.269396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6742ce-fc78-4fe0-9e75-1baa40ee03f9 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.269601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6742ce-fc78-4fe0-9e75-1baa40ee03f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.298286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0a5db2c2-5fd2-4315-9da0-1e54a4d402c1 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:06.304768 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0a5db2c2-5fd2-4315-9da0-1e54a4d402c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:06.309352 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7d6742ce-fc78-4fe0-9e75-1baa40ee03f9 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:06.311022 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7d6742ce-fc78-4fe0-9e75-1baa40ee03f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:06.326073 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 768/1541] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:06.330458 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5db2c2-5fd2-4315-9da0-1e54a4d402c1 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:06.332540 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 774/1542] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:06.348615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5db2c2-5fd2-4315-9da0-1e54a4d402c1 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:06.353926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e4f4a-6fac-482a-8f90-74964289ebc6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.354370 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e4f4a-6fac-482a-8f90-74964289ebc6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.354487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e4f4a-6fac-482a-8f90-74964289ebc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.371597 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad2de4-0d4c-4c97-9042-a44268cfb387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:06.372264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad2de4-0d4c-4c97-9042-a44268cfb387 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.372544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad2de4-0d4c-4c97-9042-a44268cfb387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.372818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad2de4-0d4c-4c97-9042-a44268cfb387 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.373511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad2de4-0d4c-4c97-9042-a44268cfb387 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:14:06.373816 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad2de4-0d4c-4c97-9042-a44268cfb387 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:06.375318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: WARNING py.warnings [None req-52ad2de4-0d4c-4c97-9042-a44268cfb387 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:14:06.375318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: warnings.warn(msg) Sep 06 16:14:06.375318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: Sep 06 16:14:06.376227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad2de4-0d4c-4c97-9042-a44268cfb387 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:06.390272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f54e4f4a-6fac-482a-8f90-74964289ebc6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:06.391559 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f54e4f4a-6fac-482a-8f90-74964289ebc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:06.403816 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 769/1543] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:14:06.413346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 775/1544] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:06.413580 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad2de4-0d4c-4c97-9042-a44268cfb387 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:06.425186 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad2de4-0d4c-4c97-9042-a44268cfb387 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:06.430222 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e4f4a-6fac-482a-8f90-74964289ebc6 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:06.446643 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a719a28-ff07-4afc-a6fc-6e3c25ca358d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:06.447258 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e4f4a-6fac-482a-8f90-74964289ebc6 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:06.447405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719a28-ff07-4afc-a6fc-6e3c25ca358d admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.447585 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719a28-ff07-4afc-a6fc-6e3c25ca358d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.447826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719a28-ff07-4afc-a6fc-6e3c25ca358d admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.448571 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a719a28-ff07-4afc-a6fc-6e3c25ca358d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:06.450495 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a719a28-ff07-4afc-a6fc-6e3c25ca358d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:06.467011 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae6709-8836-494b-8c3a-d82e2659977f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:06.467554 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae6709-8836-494b-8c3a-d82e2659977f admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.467835 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae6709-8836-494b-8c3a-d82e2659977f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.468110 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae6709-8836-494b-8c3a-d82e2659977f admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.468768 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae6709-8836-494b-8c3a-d82e2659977f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:14:06.469074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae6709-8836-494b-8c3a-d82e2659977f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:06.474713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae6709-8836-494b-8c3a-d82e2659977f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:06.502922 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 776/1545] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:14:06.512271 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae6709-8836-494b-8c3a-d82e2659977f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:06.534350 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 770/1546] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:06 2022] POST /identity/v3/projects => generated 369 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:06.535317 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae6709-8836-494b-8c3a-d82e2659977f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:06.539150 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a719a28-ff07-4afc-a6fc-6e3c25ca358d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:06.664810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e230428-530e-493f-b6bf-833a8f521e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:06.665444 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e230428-530e-493f-b6bf-833a8f521e9b admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.665721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e230428-530e-493f-b6bf-833a8f521e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.665991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e230428-530e-493f-b6bf-833a8f521e9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.666652 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e230428-530e-493f-b6bf-833a8f521e9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:06.667709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: WARNING py.warnings [None req-8e230428-530e-493f-b6bf-833a8f521e9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:14:06.667709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: warnings.warn(msg) Sep 06 16:14:06.667709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: Sep 06 16:14:06.668941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e230428-530e-493f-b6bf-833a8f521e9b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:06.688073 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a719a28-ff07-4afc-a6fc-6e3c25ca358d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:06.702087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72752ada-8f65-40ca-b7fb-91ab50ad71a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:06.702731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72752ada-8f65-40ca-b7fb-91ab50ad71a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.703361 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72752ada-8f65-40ca-b7fb-91ab50ad71a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.703652 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72752ada-8f65-40ca-b7fb-91ab50ad71a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.704483 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72752ada-8f65-40ca-b7fb-91ab50ad71a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:06.706303 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: WARNING py.warnings [None req-72752ada-8f65-40ca-b7fb-91ab50ad71a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:14:06.706303 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: warnings.warn(msg) Sep 06 16:14:06.706303 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: Sep 06 16:14:06.707709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72752ada-8f65-40ca-b7fb-91ab50ad71a0 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:06.727739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 777/1547] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:06 2022] POST /identity/v3/projects => generated 371 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:14:06.729397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7187db12-f104-4e5c-87f8-2349192491b0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.730450 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7187db12-f104-4e5c-87f8-2349192491b0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.730450 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7187db12-f104-4e5c-87f8-2349192491b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.753617 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 771/1548] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:06 2022] POST /identity/v3/users => generated 285 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:06.753617 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd7a0a-b82d-4a35-824d-42e520d43f35 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.753617 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd7a0a-b82d-4a35-824d-42e520d43f35 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.753617 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd7a0a-b82d-4a35-824d-42e520d43f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.756705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7187db12-f104-4e5c-87f8-2349192491b0 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:06.758082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7187db12-f104-4e5c-87f8-2349192491b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:06.773364 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-19cd7a0a-b82d-4a35-824d-42e520d43f35 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:06.774717 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-19cd7a0a-b82d-4a35-824d-42e520d43f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:06.822021 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 778/1549] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:06.823447 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ededf66-c9a4-4384-bded-b6f220af43d6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.823741 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ededf66-c9a4-4384-bded-b6f220af43d6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.824015 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ededf66-c9a4-4384-bded-b6f220af43d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.837593 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 772/1550] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:06.839211 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564bf46-a692-455f-998b-624457fb2a68 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.839499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564bf46-a692-455f-998b-624457fb2a68 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.839776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564bf46-a692-455f-998b-624457fb2a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.851738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4ededf66-c9a4-4384-bded-b6f220af43d6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:06.853039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4ededf66-c9a4-4384-bded-b6f220af43d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:06.860011 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d564bf46-a692-455f-998b-624457fb2a68 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:06.861085 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 779/1551] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:06.861276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d564bf46-a692-455f-998b-624457fb2a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:06.863121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a587f-41b5-4361-b698-7479131acff1 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.864458 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a587f-41b5-4361-b698-7479131acff1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.864458 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a587f-41b5-4361-b698-7479131acff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.875503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 773/1552] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:06.876396 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564bf46-a692-455f-998b-624457fb2a68 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:06.886227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564bf46-a692-455f-998b-624457fb2a68 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:06.888991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-409a587f-41b5-4361-b698-7479131acff1 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:06.890235 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-409a587f-41b5-4361-b698-7479131acff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:06.891071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c2829-3e34-4f6b-b68a-bef15180f5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:06.891671 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c2829-3e34-4f6b-b68a-bef15180f5fa admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.891943 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c2829-3e34-4f6b-b68a-bef15180f5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.892210 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c2829-3e34-4f6b-b68a-bef15180f5fa admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.892834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c2829-3e34-4f6b-b68a-bef15180f5fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:06.894811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c2829-3e34-4f6b-b68a-bef15180f5fa admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:06.898524 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 780/1553] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:06.902951 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18735a75-2581-48bd-9430-3adf5a0cc48e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.902951 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18735a75-2581-48bd-9430-3adf5a0cc48e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.902951 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18735a75-2581-48bd-9430-3adf5a0cc48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.922563 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 774/1554] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:06 2022] POST /identity/v3/users => generated 286 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:06.922563 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c2829-3e34-4f6b-b68a-bef15180f5fa admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:06.927709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-18735a75-2581-48bd-9430-3adf5a0cc48e None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:06.928929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-18735a75-2581-48bd-9430-3adf5a0cc48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:06.933106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c2829-3e34-4f6b-b68a-bef15180f5fa admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:06.937214 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 781/1555] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:06.938433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b67118-2605-46db-bfaa-11b33f0db967 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.938748 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b67118-2605-46db-bfaa-11b33f0db967 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.939024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b67118-2605-46db-bfaa-11b33f0db967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.945797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bdbf3a-c2cd-4a53-b31a-0a15e0e65002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:06.946432 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdbf3a-c2cd-4a53-b31a-0a15e0e65002 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.946741 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdbf3a-c2cd-4a53-b31a-0a15e0e65002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.947213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdbf3a-c2cd-4a53-b31a-0a15e0e65002 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.947667 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bdbf3a-c2cd-4a53-b31a-0a15e0e65002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:06.949276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bdbf3a-c2cd-4a53-b31a-0a15e0e65002 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:06.960228 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-44b67118-2605-46db-bfaa-11b33f0db967 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:06.961467 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-44b67118-2605-46db-bfaa-11b33f0db967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:06.962323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 775/1556] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:06.963666 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace4822-2b09-4ea4-8ca1-44ff670cf8a3 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.963951 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace4822-2b09-4ea4-8ca1-44ff670cf8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.964616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace4822-2b09-4ea4-8ca1-44ff670cf8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.969910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 782/1557] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:06.973320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca8da4-9a46-4a3e-9de4-5a80d07d7831 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:06.973723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca8da4-9a46-4a3e-9de4-5a80d07d7831 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:06.974072 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca8da4-9a46-4a3e-9de4-5a80d07d7831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:06.976749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:06.990387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9ace4822-2b09-4ea4-8ca1-44ff670cf8a3 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:06.991648 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9ace4822-2b09-4ea4-8ca1-44ff670cf8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:06.995548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-11ca8da4-9a46-4a3e-9de4-5a80d07d7831 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:06.996707 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-11ca8da4-9a46-4a3e-9de4-5a80d07d7831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:07.000162 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 776/1558] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:07.001347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17444a46-98a5-4548-bbb8-1c7f6b800980 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.001632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17444a46-98a5-4548-bbb8-1c7f6b800980 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.001908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17444a46-98a5-4548-bbb8-1c7f6b800980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.004746 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 783/1559] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:07.006020 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d522a-884d-4c75-8fd6-5ae2a1c77e9b None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.006308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d522a-884d-4c75-8fd6-5ae2a1c77e9b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.006575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d522a-884d-4c75-8fd6-5ae2a1c77e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.022103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-17444a46-98a5-4548-bbb8-1c7f6b800980 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:07.023356 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-17444a46-98a5-4548-bbb8-1c7f6b800980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:07.025622 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-826d522a-884d-4c75-8fd6-5ae2a1c77e9b None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:07.026822 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-826d522a-884d-4c75-8fd6-5ae2a1c77e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:07.031628 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 777/1560] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:07.032428 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17444a46-98a5-4548-bbb8-1c7f6b800980 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.038884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 784/1561] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:07.038884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8ca08-c099-4b92-a700-979483ee6304 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.038884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8ca08-c099-4b92-a700-979483ee6304 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.038884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8ca08-c099-4b92-a700-979483ee6304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.043707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17444a46-98a5-4548-bbb8-1c7f6b800980 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.048359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7c9f9-ff1c-4b7f-bf14-f7a1cf83ccf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.048963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7c9f9-ff1c-4b7f-bf14-f7a1cf83ccf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.049267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7c9f9-ff1c-4b7f-bf14-f7a1cf83ccf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.050793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7c9f9-ff1c-4b7f-bf14-f7a1cf83ccf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.051240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca7c9f9-ff1c-4b7f-bf14-f7a1cf83ccf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.052849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca7c9f9-ff1c-4b7f-bf14-f7a1cf83ccf1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.057432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ddf8ca08-c099-4b92-a700-979483ee6304 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:07.058625 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ddf8ca08-c099-4b92-a700-979483ee6304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:07.063327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 778/1562] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:07.063327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7c9f9-ff1c-4b7f-bf14-f7a1cf83ccf1 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.067487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 785/1563] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:07.068433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8ca08-c099-4b92-a700-979483ee6304 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.076541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7c9f9-ff1c-4b7f-bf14-f7a1cf83ccf1 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.077657 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8ca08-c099-4b92-a700-979483ee6304 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.081510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135bb11-3a29-4ebe-9ef9-8177aec1cb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.081823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135bb11-3a29-4ebe-9ef9-8177aec1cb5a admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.082057 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135bb11-3a29-4ebe-9ef9-8177aec1cb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.082329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135bb11-3a29-4ebe-9ef9-8177aec1cb5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.082443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a534b6-6e23-43a8-8575-007c2be07a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.082956 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a534b6-6e23-43a8-8575-007c2be07a66 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.083055 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135bb11-3a29-4ebe-9ef9-8177aec1cb5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.083235 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a534b6-6e23-43a8-8575-007c2be07a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.083653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a534b6-6e23-43a8-8575-007c2be07a66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.084145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a534b6-6e23-43a8-8575-007c2be07a66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:14:07.084440 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a534b6-6e23-43a8-8575-007c2be07a66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.084590 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135bb11-3a29-4ebe-9ef9-8177aec1cb5a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.086015 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a534b6-6e23-43a8-8575-007c2be07a66 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.087709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:07.095350 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 779/1564] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:07.096187 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135bb11-3a29-4ebe-9ef9-8177aec1cb5a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.102872 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 786/1565] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:14:07.104076 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a534b6-6e23-43a8-8575-007c2be07a66 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.106877 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135bb11-3a29-4ebe-9ef9-8177aec1cb5a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.111659 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ef07f-6862-4acf-886e-faf71dafe122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.113101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ef07f-6862-4acf-886e-faf71dafe122 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.113551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ef07f-6862-4acf-886e-faf71dafe122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.113833 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ef07f-6862-4acf-886e-faf71dafe122 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.114483 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ef07f-6862-4acf-886e-faf71dafe122 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:14:07.114817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ef07f-6862-4acf-886e-faf71dafe122 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.115928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a534b6-6e23-43a8-8575-007c2be07a66 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.116416 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ef07f-6862-4acf-886e-faf71dafe122 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.120803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a4045-fe1b-41ae-89db-c6b5041ca97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.121411 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a4045-fe1b-41ae-89db-c6b5041ca97c admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.121726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a4045-fe1b-41ae-89db-c6b5041ca97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.121965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a4045-fe1b-41ae-89db-c6b5041ca97c admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.122610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a4045-fe1b-41ae-89db-c6b5041ca97c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:14:07.123068 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a4045-fe1b-41ae-89db-c6b5041ca97c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.124658 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a4045-fe1b-41ae-89db-c6b5041ca97c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.128706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 780/1566] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:14:07.131611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ef07f-6862-4acf-886e-faf71dafe122 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.138690 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 787/1567] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:14:07.139610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a4045-fe1b-41ae-89db-c6b5041ca97c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.142843 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ef07f-6862-4acf-886e-faf71dafe122 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.148331 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f9ebd-f078-4d9f-9d07-15a512b3a109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.148941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f9ebd-f078-4d9f-9d07-15a512b3a109 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.149219 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f9ebd-f078-4d9f-9d07-15a512b3a109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.149490 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f9ebd-f078-4d9f-9d07-15a512b3a109 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.150684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f9ebd-f078-4d9f-9d07-15a512b3a109 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:14:07.150684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f9ebd-f078-4d9f-9d07-15a512b3a109 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.150926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a4045-fe1b-41ae-89db-c6b5041ca97c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.152155 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f9ebd-f078-4d9f-9d07-15a512b3a109 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.156188 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c5838-9a1f-4633-91c5-81bcecca4524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.156856 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c5838-9a1f-4633-91c5-81bcecca4524 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.157203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c5838-9a1f-4633-91c5-81bcecca4524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.157539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c5838-9a1f-4633-91c5-81bcecca4524 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.158257 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c5838-9a1f-4633-91c5-81bcecca4524 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:14:07.158630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c5838-9a1f-4633-91c5-81bcecca4524 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.161481 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c5838-9a1f-4633-91c5-81bcecca4524 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.164732 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 781/1568] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:14:07.165755 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f9ebd-f078-4d9f-9d07-15a512b3a109 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.175438 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f9ebd-f078-4d9f-9d07-15a512b3a109 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.177764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 788/1569] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:14:07.178531 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c5838-9a1f-4633-91c5-81bcecca4524 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.181045 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb205e-178e-4146-90ab-80c5635cddff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.181756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb205e-178e-4146-90ab-80c5635cddff admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.182106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb205e-178e-4146-90ab-80c5635cddff admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.182456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb205e-178e-4146-90ab-80c5635cddff admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.183202 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb205e-178e-4146-90ab-80c5635cddff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.185139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb205e-178e-4146-90ab-80c5635cddff admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.189479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c5838-9a1f-4633-91c5-81bcecca4524 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.195149 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2757c8c-a582-4bee-a692-e5e529bfc4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.195755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2757c8c-a582-4bee-a692-e5e529bfc4a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.196026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2757c8c-a582-4bee-a692-e5e529bfc4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.196292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 782/1570] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:07.196433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2757c8c-a582-4bee-a692-e5e529bfc4a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.197006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2757c8c-a582-4bee-a692-e5e529bfc4a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:14:07.197148 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb205e-178e-4146-90ab-80c5635cddff admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.197310 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2757c8c-a582-4bee-a692-e5e529bfc4a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.198935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2757c8c-a582-4bee-a692-e5e529bfc4a0 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.207265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb205e-178e-4146-90ab-80c5635cddff admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.213004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61be7cd-6017-473c-a62c-b5507117373e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.213632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 789/1571] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:14:07.214530 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2757c8c-a582-4bee-a692-e5e529bfc4a0 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.215101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61be7cd-6017-473c-a62c-b5507117373e admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.215401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61be7cd-6017-473c-a62c-b5507117373e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.215685 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61be7cd-6017-473c-a62c-b5507117373e admin admin] PATH_INFO: `/v3/projects/5b7d99d91ca94bca814f7c9e7b39b0a4/users/2f167e0c4fe6494daafbc8d607656d10/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.226701 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2757c8c-a582-4bee-a692-e5e529bfc4a0 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.232281 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19741eb8-78d7-45e4-b223-1cd388b483fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.232896 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19741eb8-78d7-45e4-b223-1cd388b483fa admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.233174 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19741eb8-78d7-45e4-b223-1cd388b483fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.233439 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19741eb8-78d7-45e4-b223-1cd388b483fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.234085 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19741eb8-78d7-45e4-b223-1cd388b483fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.236031 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19741eb8-78d7-45e4-b223-1cd388b483fa admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.237267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61be7cd-6017-473c-a62c-b5507117373e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7d99d91ca94bca814f7c9e7b39b0a4, user_id=2f167e0c4fe6494daafbc8d607656d10, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.240099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61be7cd-6017-473c-a62c-b5507117373e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.261794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 783/1572] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:07 2022] PUT /identity/v3/projects/5b7d99d91ca94bca814f7c9e7b39b0a4/users/2f167e0c4fe6494daafbc8d607656d10/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:07.262763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61be7cd-6017-473c-a62c-b5507117373e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.266381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 790/1573] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:07 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:07.267759 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19741eb8-78d7-45e4-b223-1cd388b483fa admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.340529 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61be7cd-6017-473c-a62c-b5507117373e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.340797 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19741eb8-78d7-45e4-b223-1cd388b483fa admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.345943 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089bce1-e3ed-40c2-a8db-b890f6148566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.346366 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a4ca9-14d8-486c-92a0-2c91905cd6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.347262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089bce1-e3ed-40c2-a8db-b890f6148566 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.347536 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089bce1-e3ed-40c2-a8db-b890f6148566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.347780 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a4ca9-14d8-486c-92a0-2c91905cd6e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.347931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089bce1-e3ed-40c2-a8db-b890f6148566 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.348043 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a4ca9-14d8-486c-92a0-2c91905cd6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.348323 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a4ca9-14d8-486c-92a0-2c91905cd6e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.348476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089bce1-e3ed-40c2-a8db-b890f6148566 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.349131 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a4ca9-14d8-486c-92a0-2c91905cd6e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.350448 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089bce1-e3ed-40c2-a8db-b890f6148566 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.351214 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a4ca9-14d8-486c-92a0-2c91905cd6e4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.382539 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 784/1574] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:07 2022] POST /identity/v3/projects => generated 367 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:07.382796 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 791/1575] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:07 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:07.382935 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089bce1-e3ed-40c2-a8db-b890f6148566 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.383032 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a4ca9-14d8-486c-92a0-2c91905cd6e4 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.438026 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:07.450898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089bce1-e3ed-40c2-a8db-b890f6148566 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.456120 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4363e0e-7c91-4698-b18e-a41858b66a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.456784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4363e0e-7c91-4698-b18e-a41858b66a98 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.456983 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4363e0e-7c91-4698-b18e-a41858b66a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.457253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4363e0e-7c91-4698-b18e-a41858b66a98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.457350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a4ca9-14d8-486c-92a0-2c91905cd6e4 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.457944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4363e0e-7c91-4698-b18e-a41858b66a98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.459998 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4363e0e-7c91-4698-b18e-a41858b66a98 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.462229 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb649ed6-b381-4cf4-924f-3e5f0415e099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.462939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb649ed6-b381-4cf4-924f-3e5f0415e099 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.463217 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb649ed6-b381-4cf4-924f-3e5f0415e099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.463489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb649ed6-b381-4cf4-924f-3e5f0415e099 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.464290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb649ed6-b381-4cf4-924f-3e5f0415e099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.466595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb649ed6-b381-4cf4-924f-3e5f0415e099 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.491020 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 785/1576] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:07 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:07.491934 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4363e0e-7c91-4698-b18e-a41858b66a98 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.494996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 792/1577] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:07 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:07.495805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb649ed6-b381-4cf4-924f-3e5f0415e099 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.500413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:07.562712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4363e0e-7c91-4698-b18e-a41858b66a98 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.568072 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209343f5-60ca-407f-ae4d-431f6ee6ce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.568745 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209343f5-60ca-407f-ae4d-431f6ee6ce7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.569021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209343f5-60ca-407f-ae4d-431f6ee6ce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.569302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209343f5-60ca-407f-ae4d-431f6ee6ce7d admin admin] PATH_INFO: `/v3/projects/347b53ac5908475abfedad60e71e9aff/users/6dba48c67531490685d7dd342a2661ce/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.569697 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb649ed6-b381-4cf4-924f-3e5f0415e099 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.574871 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483c2f4-48bd-413c-8a9b-fd4235e4e51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.575512 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483c2f4-48bd-413c-8a9b-fd4235e4e51f admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.575966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483c2f4-48bd-413c-8a9b-fd4235e4e51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.576244 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483c2f4-48bd-413c-8a9b-fd4235e4e51f admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.576888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483c2f4-48bd-413c-8a9b-fd4235e4e51f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.578870 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483c2f4-48bd-413c-8a9b-fd4235e4e51f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.584868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209343f5-60ca-407f-ae4d-431f6ee6ce7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=347b53ac5908475abfedad60e71e9aff, user_id=6dba48c67531490685d7dd342a2661ce, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.587802 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209343f5-60ca-407f-ae4d-431f6ee6ce7d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.607088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 786/1578] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:07 2022] PUT /identity/v3/projects/347b53ac5908475abfedad60e71e9aff/users/6dba48c67531490685d7dd342a2661ce/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:07.608021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209343f5-60ca-407f-ae4d-431f6ee6ce7d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.619816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 793/1579] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:07 2022] POST /identity/v3/projects => generated 393 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:07.621193 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a16a38-a31f-4df7-b470-53a08d8aeb29 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.621553 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a16a38-a31f-4df7-b470-53a08d8aeb29 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.621922 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a16a38-a31f-4df7-b470-53a08d8aeb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.652792 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-92a16a38-a31f-4df7-b470-53a08d8aeb29 None None] MFA Rules not processed for user `2f167e0c4fe6494daafbc8d607656d10`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:07.654033 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-92a16a38-a31f-4df7-b470-53a08d8aeb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:07.682494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209343f5-60ca-407f-ae4d-431f6ee6ce7d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.726329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 794/1580] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:07.727393 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a16a38-a31f-4df7-b470-53a08d8aeb29 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.743260 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d021ef-be4e-494c-b043-21b2ff4252e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.743801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d021ef-be4e-494c-b043-21b2ff4252e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.744079 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d021ef-be4e-494c-b043-21b2ff4252e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.744352 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d021ef-be4e-494c-b043-21b2ff4252e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.745046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d021ef-be4e-494c-b043-21b2ff4252e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.747002 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d021ef-be4e-494c-b043-21b2ff4252e8 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.764238 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:07.771739 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 787/1581] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:07 2022] POST /identity/v3/users => generated 302 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:07.773052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d021ef-be4e-494c-b043-21b2ff4252e8 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.783081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d021ef-be4e-494c-b043-21b2ff4252e8 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.788790 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd5f37-ba1f-4fc5-9738-bfb3ffee6e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.789543 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd5f37-ba1f-4fc5-9738-bfb3ffee6e77 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.789832 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd5f37-ba1f-4fc5-9738-bfb3ffee6e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.790111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd5f37-ba1f-4fc5-9738-bfb3ffee6e77 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.790770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd5f37-ba1f-4fc5-9738-bfb3ffee6e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.792772 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd5f37-ba1f-4fc5-9738-bfb3ffee6e77 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.796117 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a16a38-a31f-4df7-b470-53a08d8aeb29 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.801035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7fda5-ccd6-4448-a3da-61a1a6fa9b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.801641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7fda5-ccd6-4448-a3da-61a1a6fa9b51 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.801916 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7fda5-ccd6-4448-a3da-61a1a6fa9b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.802187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7fda5-ccd6-4448-a3da-61a1a6fa9b51 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.802878 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7fda5-ccd6-4448-a3da-61a1a6fa9b51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.804808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7fda5-ccd6-4448-a3da-61a1a6fa9b51 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.817065 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 788/1582] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:07 2022] POST /identity/v3/users => generated 291 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:07.818008 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd5f37-ba1f-4fc5-9738-bfb3ffee6e77 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.828883 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd5f37-ba1f-4fc5-9738-bfb3ffee6e77 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.833912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 795/1583] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:07 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:07.833912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7fda5-ccd6-4448-a3da-61a1a6fa9b51 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.837658 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69e974-8df7-4659-b932-271e6acbc369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.838285 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69e974-8df7-4659-b932-271e6acbc369 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.838566 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69e974-8df7-4659-b932-271e6acbc369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.838968 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69e974-8df7-4659-b932-271e6acbc369 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.839590 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69e974-8df7-4659-b932-271e6acbc369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.841677 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69e974-8df7-4659-b932-271e6acbc369 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.841814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7fda5-ccd6-4448-a3da-61a1a6fa9b51 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.847825 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d31993-c71c-493e-8697-3ac814dcc950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.848503 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d31993-c71c-493e-8697-3ac814dcc950 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.848870 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d31993-c71c-493e-8697-3ac814dcc950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.849117 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d31993-c71c-493e-8697-3ac814dcc950 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.849784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d31993-c71c-493e-8697-3ac814dcc950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.851907 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d31993-c71c-493e-8697-3ac814dcc950 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.860291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:07.866869 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 789/1584] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:07 2022] POST /identity/v3/users => generated 293 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:07.868347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e36b5f-2501-4017-8fe8-c90195d5c280 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.868628 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e36b5f-2501-4017-8fe8-c90195d5c280 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.868900 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e36b5f-2501-4017-8fe8-c90195d5c280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.883628 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 796/1585] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:07 2022] POST /identity/v3/users => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:07.883628 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d31993-c71c-493e-8697-3ac814dcc950 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.895061 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d31993-c71c-493e-8697-3ac814dcc950 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.900495 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d9d01-a6b8-48fa-83c0-11237ae91c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.901640 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d9d01-a6b8-48fa-83c0-11237ae91c79 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.901922 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d9d01-a6b8-48fa-83c0-11237ae91c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.902241 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d9d01-a6b8-48fa-83c0-11237ae91c79 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.903132 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d9d01-a6b8-48fa-83c0-11237ae91c79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.905714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d9d01-a6b8-48fa-83c0-11237ae91c79 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.906954 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-20e36b5f-2501-4017-8fe8-c90195d5c280 None None] MFA Rules not processed for user `6dba48c67531490685d7dd342a2661ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:07.908200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-20e36b5f-2501-4017-8fe8-c90195d5c280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:07.931156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 797/1586] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:07 2022] POST /identity/v3/users => generated 297 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:07.931988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d9d01-a6b8-48fa-83c0-11237ae91c79 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.943790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d9d01-a6b8-48fa-83c0-11237ae91c79 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:07.949842 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e7086-f82b-4f44-a3ed-4024f18821a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:07.950470 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e7086-f82b-4f44-a3ed-4024f18821a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:07.950783 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e7086-f82b-4f44-a3ed-4024f18821a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:07.951059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e7086-f82b-4f44-a3ed-4024f18821a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:07.951695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e7086-f82b-4f44-a3ed-4024f18821a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:07.953617 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e7086-f82b-4f44-a3ed-4024f18821a5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:07.980189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 790/1587] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:07.981058 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e36b5f-2501-4017-8fe8-c90195d5c280 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.985333 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 798/1588] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:07 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:07.987553 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e7086-f82b-4f44-a3ed-4024f18821a5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:07.995637 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e36b5f-2501-4017-8fe8-c90195d5c280 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.062820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e7086-f82b-4f44-a3ed-4024f18821a5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.066390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3836a-e8fb-481f-9f8b-f227138264bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.067044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3836a-e8fb-481f-9f8b-f227138264bc admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.067331 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3836a-e8fb-481f-9f8b-f227138264bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.067607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3836a-e8fb-481f-9f8b-f227138264bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.068250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3836a-e8fb-481f-9f8b-f227138264bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.068532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc63c0c-e503-4843-b2d4-e3b7e0a0c35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.069141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc63c0c-e503-4843-b2d4-e3b7e0a0c35b admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.069427 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc63c0c-e503-4843-b2d4-e3b7e0a0c35b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.069698 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc63c0c-e503-4843-b2d4-e3b7e0a0c35b admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.069886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3836a-e8fb-481f-9f8b-f227138264bc admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.070345 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc63c0c-e503-4843-b2d4-e3b7e0a0c35b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.071986 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc63c0c-e503-4843-b2d4-e3b7e0a0c35b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.080550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 791/1589] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:08.086131 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 799/1590] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:08.086131 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc63c0c-e503-4843-b2d4-e3b7e0a0c35b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.086408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3836a-e8fb-481f-9f8b-f227138264bc admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.094236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc63c0c-e503-4843-b2d4-e3b7e0a0c35b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.094851 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3836a-e8fb-481f-9f8b-f227138264bc admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.099610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f39c9-0454-4239-aac1-b164cffd9cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.100116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c39fdc-5f88-41fe-a5f7-3644b5161251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.100258 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f39c9-0454-4239-aac1-b164cffd9cdb admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.100506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f39c9-0454-4239-aac1-b164cffd9cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.100710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c39fdc-5f88-41fe-a5f7-3644b5161251 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.100817 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f39c9-0454-4239-aac1-b164cffd9cdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.100991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c39fdc-5f88-41fe-a5f7-3644b5161251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.101265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c39fdc-5f88-41fe-a5f7-3644b5161251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.101382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f39c9-0454-4239-aac1-b164cffd9cdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.101895 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c39fdc-5f88-41fe-a5f7-3644b5161251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.103001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f39c9-0454-4239-aac1-b164cffd9cdb admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.103506 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c39fdc-5f88-41fe-a5f7-3644b5161251 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.113320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 800/1591] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:08.116891 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 792/1592] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:08.117075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f39c9-0454-4239-aac1-b164cffd9cdb admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.117180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c39fdc-5f88-41fe-a5f7-3644b5161251 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.125301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c39fdc-5f88-41fe-a5f7-3644b5161251 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.126032 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f39c9-0454-4239-aac1-b164cffd9cdb admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.130009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fcf7c2-b57f-4c99-a21f-40aba9e70c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.130644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcf7c2-b57f-4c99-a21f-40aba9e70c4c admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.130777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4d7a3-f353-47c7-aa24-57c2afe3a9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.130950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcf7c2-b57f-4c99-a21f-40aba9e70c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.131222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcf7c2-b57f-4c99-a21f-40aba9e70c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.131406 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4d7a3-f353-47c7-aa24-57c2afe3a9bd admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.131666 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4d7a3-f353-47c7-aa24-57c2afe3a9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.131858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fcf7c2-b57f-4c99-a21f-40aba9e70c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.131965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4d7a3-f353-47c7-aa24-57c2afe3a9bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.132559 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4d7a3-f353-47c7-aa24-57c2afe3a9bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.133585 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fcf7c2-b57f-4c99-a21f-40aba9e70c4c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.134142 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4d7a3-f353-47c7-aa24-57c2afe3a9bd admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.145282 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 793/1593] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:08.145805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 801/1594] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:08.146852 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fcf7c2-b57f-4c99-a21f-40aba9e70c4c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.147359 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d87728-4d00-4ee0-99c3-46d20401cfb2 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.147661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d87728-4d00-4ee0-99c3-46d20401cfb2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.147922 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d87728-4d00-4ee0-99c3-46d20401cfb2 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.149339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 802/1595] 10.222.0.26 () {54 vars in 962 bytes} [Tue Sep 6 16:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:14:08.150100 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d87728-4d00-4ee0-99c3-46d20401cfb2 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.158561 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fcf7c2-b57f-4c99-a21f-40aba9e70c4c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.163234 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d87728-4d00-4ee0-99c3-46d20401cfb2 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.164117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4915e8-e4fa-4383-a194-8fe6330e8c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.164646 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4915e8-e4fa-4383-a194-8fe6330e8c84 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.164889 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4915e8-e4fa-4383-a194-8fe6330e8c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.165159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4915e8-e4fa-4383-a194-8fe6330e8c84 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.165782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4915e8-e4fa-4383-a194-8fe6330e8c84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.167733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4915e8-e4fa-4383-a194-8fe6330e8c84 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.168067 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78333dc1-e52f-4f21-a8ae-efa63fbc1a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.168656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78333dc1-e52f-4f21-a8ae-efa63fbc1a4a admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.168935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78333dc1-e52f-4f21-a8ae-efa63fbc1a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.169223 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78333dc1-e52f-4f21-a8ae-efa63fbc1a4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.169850 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78333dc1-e52f-4f21-a8ae-efa63fbc1a4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.171462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78333dc1-e52f-4f21-a8ae-efa63fbc1a4a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.183105 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 803/1596] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:08.187369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78333dc1-e52f-4f21-a8ae-efa63fbc1a4a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.194621 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 794/1597] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:08 2022] POST /identity/v3/users => generated 292 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:08.194621 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4915e8-e4fa-4383-a194-8fe6330e8c84 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.198343 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78333dc1-e52f-4f21-a8ae-efa63fbc1a4a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.202989 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1849a1-ed84-4a7a-80db-4806460a1670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.203585 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4915e8-e4fa-4383-a194-8fe6330e8c84 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.203734 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1849a1-ed84-4a7a-80db-4806460a1670 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.203894 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1849a1-ed84-4a7a-80db-4806460a1670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.204128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1849a1-ed84-4a7a-80db-4806460a1670 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.204759 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1849a1-ed84-4a7a-80db-4806460a1670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.206451 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1849a1-ed84-4a7a-80db-4806460a1670 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.209372 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4c094-144f-4183-8dac-a313057b5295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.210004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4c094-144f-4183-8dac-a313057b5295 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.210438 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4c094-144f-4183-8dac-a313057b5295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.210744 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4c094-144f-4183-8dac-a313057b5295 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.211417 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4c094-144f-4183-8dac-a313057b5295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.213003 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4c094-144f-4183-8dac-a313057b5295 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.215262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:08.216798 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 804/1598] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:08.217624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1849a1-ed84-4a7a-80db-4806460a1670 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.229337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 795/1599] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:08.229337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4c094-144f-4183-8dac-a313057b5295 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.229736 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1849a1-ed84-4a7a-80db-4806460a1670 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.233109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c8fe5-0768-4610-a246-36b2d97dc58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.233746 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c8fe5-0768-4610-a246-36b2d97dc58b admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.234025 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c8fe5-0768-4610-a246-36b2d97dc58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.234290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c8fe5-0768-4610-a246-36b2d97dc58b admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.234947 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c8fe5-0768-4610-a246-36b2d97dc58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.236682 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c8fe5-0768-4610-a246-36b2d97dc58b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.240923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4c094-144f-4183-8dac-a313057b5295 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.245559 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3eb60-dbaa-4928-9b31-8ac039f62cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.246232 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3eb60-dbaa-4928-9b31-8ac039f62cbb admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.246642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3eb60-dbaa-4928-9b31-8ac039f62cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.246947 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3eb60-dbaa-4928-9b31-8ac039f62cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.248122 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3eb60-dbaa-4928-9b31-8ac039f62cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.248367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 805/1600] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:08.249187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c8fe5-0768-4610-a246-36b2d97dc58b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.249739 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3eb60-dbaa-4928-9b31-8ac039f62cbb admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.260919 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c8fe5-0768-4610-a246-36b2d97dc58b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.266342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 796/1601] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:08.266342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd23ac3-5530-4198-b620-698845a7ad64 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.266342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd23ac3-5530-4198-b620-698845a7ad64 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.266342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd23ac3-5530-4198-b620-698845a7ad64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.270180 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686c385-fff6-4453-948b-830b727b8962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.270895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686c385-fff6-4453-948b-830b727b8962 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.271168 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686c385-fff6-4453-948b-830b727b8962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.271440 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686c385-fff6-4453-948b-830b727b8962 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.272116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686c385-fff6-4453-948b-830b727b8962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.274038 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686c385-fff6-4453-948b-830b727b8962 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.288705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 806/1602] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:08.288705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686c385-fff6-4453-948b-830b727b8962 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.298860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686c385-fff6-4453-948b-830b727b8962 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.303892 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036ba9a-e190-4705-ab0e-ebc1bd2bae34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.304545 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036ba9a-e190-4705-ab0e-ebc1bd2bae34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.304820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036ba9a-e190-4705-ab0e-ebc1bd2bae34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.305090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036ba9a-e190-4705-ab0e-ebc1bd2bae34 admin admin] PATH_INFO: `/v3/projects/edebb17851ef400fb71e07c088a163fb/users/b711d5b6bfa94155b9ccd846c6d7222e/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.309092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-cbd23ac3-5530-4198-b620-698845a7ad64 None None] MFA Rules not processed for user `6afd4f6ffdae4b16aa5ba757e68c18c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:08.310330 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cbd23ac3-5530-4198-b620-698845a7ad64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:08.322381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7036ba9a-e190-4705-ab0e-ebc1bd2bae34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edebb17851ef400fb71e07c088a163fb, user_id=b711d5b6bfa94155b9ccd846c6d7222e, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.323452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: WARNING py.warnings [None req-7036ba9a-e190-4705-ab0e-ebc1bd2bae34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:14:08.323452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: warnings.warn(msg) Sep 06 16:14:08.323452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: Sep 06 16:14:08.325607 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7036ba9a-e190-4705-ab0e-ebc1bd2bae34 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.341146 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 807/1603] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:08 2022] PUT /identity/v3/projects/edebb17851ef400fb71e07c088a163fb/users/b711d5b6bfa94155b9ccd846c6d7222e/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:08.342215 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036ba9a-e190-4705-ab0e-ebc1bd2bae34 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.372844 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:08.386006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 797/1604] 10.222.0.26 () {58 vars in 1059 bytes} [Tue Sep 6 16:14:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:08.386889 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd23ac3-5530-4198-b620-698845a7ad64 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.399343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd23ac3-5530-4198-b620-698845a7ad64 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.423819 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036ba9a-e190-4705-ab0e-ebc1bd2bae34 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.429276 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3d8e4-3a20-45cb-a33d-8e25bd382222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.429883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3d8e4-3a20-45cb-a33d-8e25bd382222 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.430173 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3d8e4-3a20-45cb-a33d-8e25bd382222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.430444 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3d8e4-3a20-45cb-a33d-8e25bd382222 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.431121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3d8e4-3a20-45cb-a33d-8e25bd382222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.433450 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3d8e4-3a20-45cb-a33d-8e25bd382222 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.448408 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 808/1605] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:08.449289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3d8e4-3a20-45cb-a33d-8e25bd382222 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.460690 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3d8e4-3a20-45cb-a33d-8e25bd382222 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.461604 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99efb8-8483-42aa-a07d-19aa35e94ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.462271 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99efb8-8483-42aa-a07d-19aa35e94ae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.462565 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99efb8-8483-42aa-a07d-19aa35e94ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.462907 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99efb8-8483-42aa-a07d-19aa35e94ae3 admin admin] PATH_INFO: `/v3/projects/d306388a489a4b42a98a85e03968dba7/users/95ac6a289fb345d3a98871f5e11f9414/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.465325 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c7350-f9d3-431f-8f55-725718527657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.465983 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c7350-f9d3-431f-8f55-725718527657 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.466260 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c7350-f9d3-431f-8f55-725718527657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.466531 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c7350-f9d3-431f-8f55-725718527657 admin admin] PATH_INFO: `/v3/projects/43dcc617cde94adf87bfa665e6cfa39c/users/c7e629d1f7ec43d8af4fbff93b6bffad/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.478749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f99efb8-8483-42aa-a07d-19aa35e94ae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d306388a489a4b42a98a85e03968dba7, user_id=95ac6a289fb345d3a98871f5e11f9414, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.481449 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f99efb8-8483-42aa-a07d-19aa35e94ae3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.482551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c7350-f9d3-431f-8f55-725718527657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43dcc617cde94adf87bfa665e6cfa39c, user_id=c7e629d1f7ec43d8af4fbff93b6bffad, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.485900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c7350-f9d3-431f-8f55-725718527657 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.500725 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 798/1606] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:08 2022] PUT /identity/v3/projects/d306388a489a4b42a98a85e03968dba7/users/95ac6a289fb345d3a98871f5e11f9414/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:08.500725 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99efb8-8483-42aa-a07d-19aa35e94ae3 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.501104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 809/1607] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:08 2022] PUT /identity/v3/projects/43dcc617cde94adf87bfa665e6cfa39c/users/c7e629d1f7ec43d8af4fbff93b6bffad/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:08.501916 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c7350-f9d3-431f-8f55-725718527657 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.561587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:08.574587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c7350-f9d3-431f-8f55-725718527657 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.579530 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabfd6e-b417-498a-ae3a-d81ef5b26290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.580125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99efb8-8483-42aa-a07d-19aa35e94ae3 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.580246 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabfd6e-b417-498a-ae3a-d81ef5b26290 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.580443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabfd6e-b417-498a-ae3a-d81ef5b26290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.580786 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabfd6e-b417-498a-ae3a-d81ef5b26290 admin admin] PATH_INFO: `/v3/projects/c5974f0b22a7493da25b0b0aaa21c66b/users/668452994ad942069d30f4bddbba15b4/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.585859 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90966a9-645f-43e8-a28e-9be0b2b428f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.586526 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90966a9-645f-43e8-a28e-9be0b2b428f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.586867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90966a9-645f-43e8-a28e-9be0b2b428f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.587148 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90966a9-645f-43e8-a28e-9be0b2b428f8 admin admin] PATH_INFO: `/v3/projects/8eb61062f4ab48a49ee3b5f8d02ce761/users/12d8f855c8fc4dab9d97afc6f7bab933/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.595743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceabfd6e-b417-498a-ae3a-d81ef5b26290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5974f0b22a7493da25b0b0aaa21c66b, user_id=668452994ad942069d30f4bddbba15b4, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.599496 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceabfd6e-b417-498a-ae3a-d81ef5b26290 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.606597 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90966a9-645f-43e8-a28e-9be0b2b428f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb61062f4ab48a49ee3b5f8d02ce761, user_id=12d8f855c8fc4dab9d97afc6f7bab933, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.609826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90966a9-645f-43e8-a28e-9be0b2b428f8 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.614650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 810/1608] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:08 2022] PUT /identity/v3/projects/c5974f0b22a7493da25b0b0aaa21c66b/users/668452994ad942069d30f4bddbba15b4/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:08.615490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabfd6e-b417-498a-ae3a-d81ef5b26290 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.628338 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 799/1609] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:08 2022] PUT /identity/v3/projects/8eb61062f4ab48a49ee3b5f8d02ce761/users/12d8f855c8fc4dab9d97afc6f7bab933/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:08.629588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c3a3f-0b8c-413a-8709-1c51310050b1 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.629878 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c3a3f-0b8c-413a-8709-1c51310050b1 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.630143 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c3a3f-0b8c-413a-8709-1c51310050b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.666561 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-a52c3a3f-0b8c-413a-8709-1c51310050b1 None None] MFA Rules not processed for user `b711d5b6bfa94155b9ccd846c6d7222e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:08.667817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a52c3a3f-0b8c-413a-8709-1c51310050b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:08.679825 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabfd6e-b417-498a-ae3a-d81ef5b26290 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.692420 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:08.738927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 800/1610] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:08 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:08.739982 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c3a3f-0b8c-413a-8709-1c51310050b1 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.741487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0a506-eb5a-4790-b5f4-460e308c567e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.742160 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0a506-eb5a-4790-b5f4-460e308c567e admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.742778 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0a506-eb5a-4790-b5f4-460e308c567e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.743116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0a506-eb5a-4790-b5f4-460e308c567e admin admin] PATH_INFO: `/v3/projects/6d1e05e9e4a940e78a43d397f0c0db16/users/eda6839efaab4317a0eda1f47f1656a2/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.760599 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0a506-eb5a-4790-b5f4-460e308c567e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d1e05e9e4a940e78a43d397f0c0db16, user_id=eda6839efaab4317a0eda1f47f1656a2, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.763217 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0a506-eb5a-4790-b5f4-460e308c567e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.778501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 811/1611] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:08 2022] PUT /identity/v3/projects/6d1e05e9e4a940e78a43d397f0c0db16/users/eda6839efaab4317a0eda1f47f1656a2/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:08.779527 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0a506-eb5a-4790-b5f4-460e308c567e admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:08.817364 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c3a3f-0b8c-413a-8709-1c51310050b1 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.848426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0a506-eb5a-4790-b5f4-460e308c567e admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:08.853562 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8add466-4feb-4c57-8eca-99d59d6f4954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.854226 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8add466-4feb-4c57-8eca-99d59d6f4954 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.854502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8add466-4feb-4c57-8eca-99d59d6f4954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.854804 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8add466-4feb-4c57-8eca-99d59d6f4954 admin admin] PATH_INFO: `/v3/projects/71e74ae793254faeafcae5fffff258e9/users/30f851e0460f408981dc139d502a4647/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.871856 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8add466-4feb-4c57-8eca-99d59d6f4954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e74ae793254faeafcae5fffff258e9, user_id=30f851e0460f408981dc139d502a4647, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.872770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:08.874737 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8add466-4feb-4c57-8eca-99d59d6f4954 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.886595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60013ebf-e45b-4859-bfbe-fb2139cc123b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:08.887341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60013ebf-e45b-4859-bfbe-fb2139cc123b service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.887692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60013ebf-e45b-4859-bfbe-fb2139cc123b service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.888146 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60013ebf-e45b-4859-bfbe-fb2139cc123b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.889902 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 812/1612] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:08 2022] PUT /identity/v3/projects/71e74ae793254faeafcae5fffff258e9/users/30f851e0460f408981dc139d502a4647/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:08.891260 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c81730-390c-40e7-91d1-dde0c92d222e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.891540 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c81730-390c-40e7-91d1-dde0c92d222e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.891826 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c81730-390c-40e7-91d1-dde0c92d222e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.913283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e9c81730-390c-40e7-91d1-dde0c92d222e None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:08.914493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e9c81730-390c-40e7-91d1-dde0c92d222e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:08.951696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60013ebf-e45b-4859-bfbe-fb2139cc123b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:08.953313 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60013ebf-e45b-4859-bfbe-fb2139cc123b service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:08.964065 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 801/1613] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:08.968103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abece68-36e9-4f3a-a65e-08b18d81b1d0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.968463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abece68-36e9-4f3a-a65e-08b18d81b1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.968852 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abece68-36e9-4f3a-a65e-08b18d81b1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:08.982846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 813/1614] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:08.985473 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe90a4-c857-4cdb-9fb0-3c2e156d7c36 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:08.986144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe90a4-c857-4cdb-9fb0-3c2e156d7c36 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:08.986652 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe90a4-c857-4cdb-9fb0-3c2e156d7c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.001688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9abece68-36e9-4f3a-a65e-08b18d81b1d0 None None] MFA Rules not processed for user `c7e629d1f7ec43d8af4fbff93b6bffad`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:09.003016 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9abece68-36e9-4f3a-a65e-08b18d81b1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:09.008779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a2fe90a4-c857-4cdb-9fb0-3c2e156d7c36 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:09.010142 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a2fe90a4-c857-4cdb-9fb0-3c2e156d7c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:09.018588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 814/1615] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:09.020629 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a9de6-af90-4a2c-9e96-6fd7527a9aae None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:09.021016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a9de6-af90-4a2c-9e96-6fd7527a9aae None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:09.021402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a9de6-af90-4a2c-9e96-6fd7527a9aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.052031 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-fe6a9de6-af90-4a2c-9e96-6fd7527a9aae None None] MFA Rules not processed for user `12d8f855c8fc4dab9d97afc6f7bab933`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:09.053686 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fe6a9de6-af90-4a2c-9e96-6fd7527a9aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:09.059412 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:09.083810 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 802/1616] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:09.084765 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abece68-36e9-4f3a-a65e-08b18d81b1d0 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:09.097471 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abece68-36e9-4f3a-a65e-08b18d81b1d0 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:09.105046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3496f9-f36f-4d4f-938d-6994244138df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:09.106717 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3496f9-f36f-4d4f-938d-6994244138df admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:09.106973 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3496f9-f36f-4d4f-938d-6994244138df admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:09.107251 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3496f9-f36f-4d4f-938d-6994244138df admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.107897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3496f9-f36f-4d4f-938d-6994244138df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:09.111060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3496f9-f36f-4d4f-938d-6994244138df admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:09.146653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 815/1617] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:09.152285 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70c609-c9ef-4632-9ea6-3f4f6e0c50b5 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:09.152986 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70c609-c9ef-4632-9ea6-3f4f6e0c50b5 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:09.153224 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70c609-c9ef-4632-9ea6-3f4f6e0c50b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.154821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 803/1618] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:09 2022] POST /identity/v3/projects => generated 403 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:09.155945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3496f9-f36f-4d4f-938d-6994244138df admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:09.173441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-0d70c609-c9ef-4632-9ea6-3f4f6e0c50b5 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:09.174773 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0d70c609-c9ef-4632-9ea6-3f4f6e0c50b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:09.228057 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3496f9-f36f-4d4f-938d-6994244138df admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:09.232819 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8b162-c4f3-434e-9aa1-e211a7af67da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:09.233426 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8b162-c4f3-434e-9aa1-e211a7af67da admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:09.233701 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8b162-c4f3-434e-9aa1-e211a7af67da admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:09.233966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8b162-c4f3-434e-9aa1-e211a7af67da admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.234598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8b162-c4f3-434e-9aa1-e211a7af67da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:09.236603 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8b162-c4f3-434e-9aa1-e211a7af67da admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:09.243029 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:09.244940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 816/1619] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:09.245846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70c609-c9ef-4632-9ea6-3f4f6e0c50b5 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:09.254473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 804/1620] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:09.255941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8b162-c4f3-434e-9aa1-e211a7af67da admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:09.258304 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-00c8b162-c4f3-434e-9aa1-e211a7af67da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:09.310146 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70c609-c9ef-4632-9ea6-3f4f6e0c50b5 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:09.314558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c53c1-6a43-4cc8-abd6-11c6d7da53f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:09.315757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c53c1-6a43-4cc8-abd6-11c6d7da53f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:09.316108 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c53c1-6a43-4cc8-abd6-11c6d7da53f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:09.316499 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c53c1-6a43-4cc8-abd6-11c6d7da53f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.320055 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8b162-c4f3-434e-9aa1-e211a7af67da admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:09.323742 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66e940-a794-4861-94a2-f9463d85491e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:09.324346 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66e940-a794-4861-94a2-f9463d85491e service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:09.324621 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66e940-a794-4861-94a2-f9463d85491e service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:09.324890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66e940-a794-4861-94a2-f9463d85491e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.326000 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c53c1-6a43-4cc8-abd6-11c6d7da53f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:09.327488 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c53c1-6a43-4cc8-abd6-11c6d7da53f6 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:09.335861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66e940-a794-4861-94a2-f9463d85491e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:09.337443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66e940-a794-4861-94a2-f9463d85491e service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:09.338841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 817/1621] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:09.340116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24198327-1623-46fa-843b-960c536e9719 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:09.340403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24198327-1623-46fa-843b-960c536e9719 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:09.340700 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24198327-1623-46fa-843b-960c536e9719 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.342200 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 818/1622] 10.222.0.26 () {54 vars in 962 bytes} [Tue Sep 6 16:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:14:09.342923 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24198327-1623-46fa-843b-960c536e9719 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:09.355931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 805/1623] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:09.357015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66e940-a794-4861-94a2-f9463d85491e service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:09.359571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24198327-1623-46fa-843b-960c536e9719 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:09.366225 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797e5b3-c19e-4d50-9fd8-e39024b2f52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:09.366701 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797e5b3-c19e-4d50-9fd8-e39024b2f52c admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:09.367401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797e5b3-c19e-4d50-9fd8-e39024b2f52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:09.367793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797e5b3-c19e-4d50-9fd8-e39024b2f52c admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.369510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797e5b3-c19e-4d50-9fd8-e39024b2f52c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:09.373672 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797e5b3-c19e-4d50-9fd8-e39024b2f52c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:09.404661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 819/1624] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:09 2022] POST /identity/v3/users => generated 308 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:09.406296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c23f36-77c2-4a09-bc7a-5bb41a6000be None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:09.406621 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c23f36-77c2-4a09-bc7a-5bb41a6000be None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:09.407007 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c23f36-77c2-4a09-bc7a-5bb41a6000be None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.409151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 820/1625] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 6 16:14:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:14:09.410124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c23f36-77c2-4a09-bc7a-5bb41a6000be None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:09.426958 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c23f36-77c2-4a09-bc7a-5bb41a6000be None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:09.434160 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11313111-6855-4428-b40d-a7e53a5c3594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:09.435177 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11313111-6855-4428-b40d-a7e53a5c3594 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:09.435655 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11313111-6855-4428-b40d-a7e53a5c3594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:09.436023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11313111-6855-4428-b40d-a7e53a5c3594 admin admin] PATH_INFO: `/v3/projects/71e74ae793254faeafcae5fffff258e9/users/30f851e0460f408981dc139d502a4647/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.439155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11313111-6855-4428-b40d-a7e53a5c3594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e74ae793254faeafcae5fffff258e9, user_id=30f851e0460f408981dc139d502a4647, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:09.443736 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11313111-6855-4428-b40d-a7e53a5c3594 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:09.444619 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66e940-a794-4861-94a2-f9463d85491e service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:09.450501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9745aa-5d9c-4f5d-8a96-40f6b3d8dc32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:09.451472 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9745aa-5d9c-4f5d-8a96-40f6b3d8dc32 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:09.451630 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9745aa-5d9c-4f5d-8a96-40f6b3d8dc32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:09.451905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9745aa-5d9c-4f5d-8a96-40f6b3d8dc32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.463952 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 821/1626] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:09 2022] PUT /identity/v3/projects/71e74ae793254faeafcae5fffff258e9/users/30f851e0460f408981dc139d502a4647/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:09.465075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11313111-6855-4428-b40d-a7e53a5c3594 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:09.501289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:09.538147 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9745aa-5d9c-4f5d-8a96-40f6b3d8dc32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:09.540141 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9745aa-5d9c-4f5d-8a96-40f6b3d8dc32 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:09.553133 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11313111-6855-4428-b40d-a7e53a5c3594 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:09.555259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 806/1627] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:09.558165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c3269-2c3d-4ae5-a46a-ec6cb26abc72 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:09.558748 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c3269-2c3d-4ae5-a46a-ec6cb26abc72 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:09.559220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c3269-2c3d-4ae5-a46a-ec6cb26abc72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.577069 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d99abd-f4b5-4c23-945d-5c29b960395a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:09.577743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d99abd-f4b5-4c23-945d-5c29b960395a service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:09.578038 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d99abd-f4b5-4c23-945d-5c29b960395a service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:09.578317 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d99abd-f4b5-4c23-945d-5c29b960395a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.592393 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-cd7c3269-2c3d-4ae5-a46a-ec6cb26abc72 None None] MFA Rules not processed for user `6afd4f6ffdae4b16aa5ba757e68c18c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:09.593709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cd7c3269-2c3d-4ae5-a46a-ec6cb26abc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:09.700308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:09.727490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d99abd-f4b5-4c23-945d-5c29b960395a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:09.729674 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d99abd-f4b5-4c23-945d-5c29b960395a service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:09.742286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 807/1628] 10.222.0.26 () {58 vars in 1059 bytes} [Tue Sep 6 16:14:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:09.743652 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c3269-2c3d-4ae5-a46a-ec6cb26abc72 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:09.749958 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 822/1629] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:09.752074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5095d4d-0cd3-496b-90f9-63dd4ffaffd4 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:09.754296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5095d4d-0cd3-496b-90f9-63dd4ffaffd4 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:09.754763 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5095d4d-0cd3-496b-90f9-63dd4ffaffd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.759402 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c3269-2c3d-4ae5-a46a-ec6cb26abc72 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:09.770640 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ba88c-6706-4c5c-a4b7-43287d5b148e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:09.771541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ba88c-6706-4c5c-a4b7-43287d5b148e admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:09.772236 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ba88c-6706-4c5c-a4b7-43287d5b148e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:09.772475 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ba88c-6706-4c5c-a4b7-43287d5b148e admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.773163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ba88c-6706-4c5c-a4b7-43287d5b148e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:09.774909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ba88c-6706-4c5c-a4b7-43287d5b148e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:09.791669 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e5095d4d-0cd3-496b-90f9-63dd4ffaffd4 None None] MFA Rules not processed for user `6afd4f6ffdae4b16aa5ba757e68c18c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:09.792990 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e5095d4d-0cd3-496b-90f9-63dd4ffaffd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:09.796759 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 808/1630] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:09.799212 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa2e64-6c7b-4c38-be0a-498b202dee67 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:09.800076 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa2e64-6c7b-4c38-be0a-498b202dee67 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:09.800567 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa2e64-6c7b-4c38-be0a-498b202dee67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.807758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 823/1631] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 6 16:14:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:09.809252 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5095d4d-0cd3-496b-90f9-63dd4ffaffd4 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:09.823995 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5095d4d-0cd3-496b-90f9-63dd4ffaffd4 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:09.833954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c61da2-f957-4033-ab4c-b794ae1dda75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:09.834920 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c61da2-f957-4033-ab4c-b794ae1dda75 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:09.835509 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c61da2-f957-4033-ab4c-b794ae1dda75 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:09.836089 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c61da2-f957-4033-ab4c-b794ae1dda75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:09.852917 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1daa2e64-6c7b-4c38-be0a-498b202dee67 None None] MFA Rules not processed for user `30f851e0460f408981dc139d502a4647`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:09.854984 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1daa2e64-6c7b-4c38-be0a-498b202dee67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:09.960715 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c61da2-f957-4033-ab4c-b794ae1dda75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:09.962403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c61da2-f957-4033-ab4c-b794ae1dda75 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:09.988120 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 824/1632] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:09.989093 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c61da2-f957-4033-ab4c-b794ae1dda75 service nova] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:10.015962 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c61da2-f957-4033-ab4c-b794ae1dda75 service nova] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:10.028407 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d93fb0-d5b3-4501-890a-708f8d514aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:10.029187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d93fb0-d5b3-4501-890a-708f8d514aa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:10.029472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d93fb0-d5b3-4501-890a-708f8d514aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:10.030872 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d93fb0-d5b3-4501-890a-708f8d514aa0 admin admin] PATH_INFO: `/v3/projects/a8db5a684e1c43ff86bb2a1fc69427ab/users/83f5c0202f7548ba9f7a35d926b0b063/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:10.035199 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 809/1633] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:10.038051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa2e64-6c7b-4c38-be0a-498b202dee67 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:10.070403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d93fb0-d5b3-4501-890a-708f8d514aa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8db5a684e1c43ff86bb2a1fc69427ab, user_id=83f5c0202f7548ba9f7a35d926b0b063, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:10.073648 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d93fb0-d5b3-4501-890a-708f8d514aa0 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:10.105325 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 825/1634] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:09 2022] PUT /identity/v3/projects/a8db5a684e1c43ff86bb2a1fc69427ab/users/83f5c0202f7548ba9f7a35d926b0b063/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:10.108361 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d93fb0-d5b3-4501-890a-708f8d514aa0 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:10.230746 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:10.236145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa2e64-6c7b-4c38-be0a-498b202dee67 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:10.290927 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d93fb0-d5b3-4501-890a-708f8d514aa0 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:10.298247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93456698-a63d-4f84-94fe-943c1668abbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:10.298899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93456698-a63d-4f84-94fe-943c1668abbf admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:10.299179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93456698-a63d-4f84-94fe-943c1668abbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:10.299488 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93456698-a63d-4f84-94fe-943c1668abbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:10.300167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93456698-a63d-4f84-94fe-943c1668abbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:10.301805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93456698-a63d-4f84-94fe-943c1668abbf admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:10.333233 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 826/1635] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:10.335116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93456698-a63d-4f84-94fe-943c1668abbf admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:10.360123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93456698-a63d-4f84-94fe-943c1668abbf admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:10.374637 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bdacd-3bdc-46f2-8797-10e369503a53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:10.375414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bdacd-3bdc-46f2-8797-10e369503a53 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:10.375829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bdacd-3bdc-46f2-8797-10e369503a53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:10.376046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bdacd-3bdc-46f2-8797-10e369503a53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:10.378409 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b2a8f-f29f-4e91-a1ac-967b86224517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:10.379120 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b2a8f-f29f-4e91-a1ac-967b86224517 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:10.379409 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b2a8f-f29f-4e91-a1ac-967b86224517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:10.379682 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b2a8f-f29f-4e91-a1ac-967b86224517 admin admin] PATH_INFO: `/v3/projects/a8db5a684e1c43ff86bb2a1fc69427ab/users/83f5c0202f7548ba9f7a35d926b0b063/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:10.382676 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b2a8f-f29f-4e91-a1ac-967b86224517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8db5a684e1c43ff86bb2a1fc69427ab, user_id=83f5c0202f7548ba9f7a35d926b0b063, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:10.385467 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b2a8f-f29f-4e91-a1ac-967b86224517 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:10.409341 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 827/1636] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:10 2022] PUT /identity/v3/projects/a8db5a684e1c43ff86bb2a1fc69427ab/users/83f5c0202f7548ba9f7a35d926b0b063/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:10.422679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79e6f4-03b9-44df-a7a8-f961b092b9e6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:10.422998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79e6f4-03b9-44df-a7a8-f961b092b9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:10.423196 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79e6f4-03b9-44df-a7a8-f961b092b9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:10.474009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:10.497873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-0e79e6f4-03b9-44df-a7a8-f961b092b9e6 None None] MFA Rules not processed for user `83f5c0202f7548ba9f7a35d926b0b063`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:10.499339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0e79e6f4-03b9-44df-a7a8-f961b092b9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:10.581828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5bdacd-3bdc-46f2-8797-10e369503a53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:10.584933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5bdacd-3bdc-46f2-8797-10e369503a53 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:10.619619 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 810/1637] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:10.620866 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bdacd-3bdc-46f2-8797-10e369503a53 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:10.635429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bdacd-3bdc-46f2-8797-10e369503a53 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:10.658726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 828/1638] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:10 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:10.660989 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaeafa5-f644-493e-bb60-c5fa125da60d None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:10.661538 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaeafa5-f644-493e-bb60-c5fa125da60d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:10.662475 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaeafa5-f644-493e-bb60-c5fa125da60d None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:10.669358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 829/1639] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 6 16:14:10 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:14:10.671578 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1771f8-1247-459e-8cba-ef0ecfaa8f84 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:10.672073 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1771f8-1247-459e-8cba-ef0ecfaa8f84 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:10.672531 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1771f8-1247-459e-8cba-ef0ecfaa8f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:10.700371 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-dd1771f8-1247-459e-8cba-ef0ecfaa8f84 None None] MFA Rules not processed for user `6afd4f6ffdae4b16aa5ba757e68c18c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:10.702422 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-dd1771f8-1247-459e-8cba-ef0ecfaa8f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:10.756951 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5ce5f-0786-47a1-b74d-fc2f159e6b8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:10.758478 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5ce5f-0786-47a1-b74d-fc2f159e6b8a service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:10.758867 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5ce5f-0786-47a1-b74d-fc2f159e6b8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:10.759228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5ce5f-0786-47a1-b74d-fc2f159e6b8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:10.846811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 830/1640] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 6 16:14:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:10.887029 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca5ce5f-0786-47a1-b74d-fc2f159e6b8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:10.888483 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca5ce5f-0786-47a1-b74d-fc2f159e6b8a service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:10.899369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 811/1641] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:11.070456 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1771f8-1247-459e-8cba-ef0ecfaa8f84 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:11.084564 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1771f8-1247-459e-8cba-ef0ecfaa8f84 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:11.091539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6031ca-89ec-4dc1-bf35-50e2c21b7305 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:11.092182 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6031ca-89ec-4dc1-bf35-50e2c21b7305 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:11.093148 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6031ca-89ec-4dc1-bf35-50e2c21b7305 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:11.093501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6031ca-89ec-4dc1-bf35-50e2c21b7305 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:11.120915 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6031ca-89ec-4dc1-bf35-50e2c21b7305 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:11.124607 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6031ca-89ec-4dc1-bf35-50e2c21b7305 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:11.153037 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 831/1642] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:11 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:11.215097 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5ce5f-0786-47a1-b74d-fc2f159e6b8a service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:11.226341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5ce5f-0786-47a1-b74d-fc2f159e6b8a service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:11.237879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1dacc-e4dd-4c5a-8268-262774bec5f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:11.238863 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1dacc-e4dd-4c5a-8268-262774bec5f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:11.239341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1dacc-e4dd-4c5a-8268-262774bec5f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:11.239827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1dacc-e4dd-4c5a-8268-262774bec5f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:11.271643 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1dacc-e4dd-4c5a-8268-262774bec5f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:11.273871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1dacc-e4dd-4c5a-8268-262774bec5f0 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:11.296306 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 812/1643] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:12.065680 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6031ca-89ec-4dc1-bf35-50e2c21b7305 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:12.096659 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6031ca-89ec-4dc1-bf35-50e2c21b7305 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:12.116781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805ef01-01ce-4668-9081-d566d5fbefe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:12.118608 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805ef01-01ce-4668-9081-d566d5fbefe1 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:12.119356 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805ef01-01ce-4668-9081-d566d5fbefe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:12.120071 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805ef01-01ce-4668-9081-d566d5fbefe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:12.197066 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:12.353922 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805ef01-01ce-4668-9081-d566d5fbefe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:12.356672 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805ef01-01ce-4668-9081-d566d5fbefe1 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:12.377250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 832/1644] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:14:12.722586 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1dacc-e4dd-4c5a-8268-262774bec5f0 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:12.937575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:12.946131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1dacc-e4dd-4c5a-8268-262774bec5f0 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:12.957231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82a63e-a150-4ac4-9386-d49f7880e94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:12.958694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82a63e-a150-4ac4-9386-d49f7880e94d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:12.959477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82a63e-a150-4ac4-9386-d49f7880e94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:12.960499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82a63e-a150-4ac4-9386-d49f7880e94d admin admin] PATH_INFO: `/v3/users/b711d5b6bfa94155b9ccd846c6d7222e` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:12.963295 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82a63e-a150-4ac4-9386-d49f7880e94d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b711d5b6bfa94155b9ccd846c6d7222e)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:12.968676 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82a63e-a150-4ac4-9386-d49f7880e94d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:13.096835 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4f82a63e-a150-4ac4-9386-d49f7880e94d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b711d5b6bfa94155b9ccd846c6d7222e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:13.121114 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4f82a63e-a150-4ac4-9386-d49f7880e94d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b711d5b6bfa94155b9ccd846c6d7222e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:13.171038 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4f82a63e-a150-4ac4-9386-d49f7880e94d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b711d5b6bfa94155b9ccd846c6d7222e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:13.172857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4f82a63e-a150-4ac4-9386-d49f7880e94d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b711d5b6bfa94155b9ccd846c6d7222e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:13.173964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4f82a63e-a150-4ac4-9386-d49f7880e94d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b711d5b6bfa94155b9ccd846c6d7222e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:13.205009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 813/1645] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:14:12 2022] DELETE /identity/v3/users/b711d5b6bfa94155b9ccd846c6d7222e => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:14:13.210337 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7202ed-8ae6-40ae-a8a2-b8de96f6026c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:13.211265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7202ed-8ae6-40ae-a8a2-b8de96f6026c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:13.211265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7202ed-8ae6-40ae-a8a2-b8de96f6026c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:13.238582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ba7202ed-8ae6-40ae-a8a2-b8de96f6026c None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:13.239852 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ba7202ed-8ae6-40ae-a8a2-b8de96f6026c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:13.392352 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 833/1646] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:13.428718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82a63e-a150-4ac4-9386-d49f7880e94d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:13.436612 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4f82a63e-a150-4ac4-9386-d49f7880e94d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:13.608448 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82a63e-a150-4ac4-9386-d49f7880e94d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:13.624070 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e06f4-06d4-4556-b977-a24f71d13606 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:13.625313 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e06f4-06d4-4556-b977-a24f71d13606 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:13.625313 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e06f4-06d4-4556-b977-a24f71d13606 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:13.625313 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e06f4-06d4-4556-b977-a24f71d13606 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:13.638394 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e06f4-06d4-4556-b977-a24f71d13606 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:13.640045 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e06f4-06d4-4556-b977-a24f71d13606 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:13.663584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 814/1647] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:15.457754 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7202ed-8ae6-40ae-a8a2-b8de96f6026c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:15.471950 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ba7202ed-8ae6-40ae-a8a2-b8de96f6026c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:15.504001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7202ed-8ae6-40ae-a8a2-b8de96f6026c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:15.544052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4c913-af7a-46df-ac5d-8434e04dfabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:15.544052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4c913-af7a-46df-ac5d-8434e04dfabf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:15.544052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4c913-af7a-46df-ac5d-8434e04dfabf admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:15.544052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4c913-af7a-46df-ac5d-8434e04dfabf admin admin] PATH_INFO: `/v3/projects/edebb17851ef400fb71e07c088a163fb` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:15.547194 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e06f4-06d4-4556-b977-a24f71d13606 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:15.547194 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2f7e06f4-06d4-4556-b977-a24f71d13606 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:15.562059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4c913-af7a-46df-ac5d-8434e04dfabf admin admin] RBAC: Authorizing `identity:delete_project(project_id=edebb17851ef400fb71e07c088a163fb)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:15.570161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4c913-af7a-46df-ac5d-8434e04dfabf admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:15.588157 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e06f4-06d4-4556-b977-a24f71d13606 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:15.609190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd5e53-77cc-4f7a-ad2f-b12eca4c62f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:15.610578 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd5e53-77cc-4f7a-ad2f-b12eca4c62f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:15.610961 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd5e53-77cc-4f7a-ad2f-b12eca4c62f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:15.611245 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd5e53-77cc-4f7a-ad2f-b12eca4c62f6 admin admin] PATH_INFO: `/v3/users/6dba48c67531490685d7dd342a2661ce` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:15.616119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd5e53-77cc-4f7a-ad2f-b12eca4c62f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dba48c67531490685d7dd342a2661ce)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:15.637386 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd5e53-77cc-4f7a-ad2f-b12eca4c62f6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:15.717040 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-34c4c913-af7a-46df-ac5d-8434e04dfabf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edebb17851ef400fb71e07c088a163fb', 'request_id': 'req-34c4c913-af7a-46df-ac5d-8434e04dfabf'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:15.729386 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-34c4c913-af7a-46df-ac5d-8434e04dfabf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edebb17851ef400fb71e07c088a163fb', 'request_id': 'req-34c4c913-af7a-46df-ac5d-8434e04dfabf'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:15.740989 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-34c4c913-af7a-46df-ac5d-8434e04dfabf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edebb17851ef400fb71e07c088a163fb', 'request_id': 'req-34c4c913-af7a-46df-ac5d-8434e04dfabf'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:15.762420 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-34c4c913-af7a-46df-ac5d-8434e04dfabf admin admin] The token cache is being invalidate because project edebb17851ef400fb71e07c088a163fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:14:15.763058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-34c4c913-af7a-46df-ac5d-8434e04dfabf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:15.764320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-34c4c913-af7a-46df-ac5d-8434e04dfabf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:15.768892 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 834/1648] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:14:15 2022] DELETE /identity/v3/projects/edebb17851ef400fb71e07c088a163fb => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:15.772813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4c913-af7a-46df-ac5d-8434e04dfabf admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:15.776280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-34c4c913-af7a-46df-ac5d-8434e04dfabf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:15.853347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a1cd5e53-77cc-4f7a-ad2f-b12eca4c62f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dba48c67531490685d7dd342a2661ce'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:15.880927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a1cd5e53-77cc-4f7a-ad2f-b12eca4c62f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dba48c67531490685d7dd342a2661ce'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:15.925740 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:15.937129 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a1cd5e53-77cc-4f7a-ad2f-b12eca4c62f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dba48c67531490685d7dd342a2661ce'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:15.940479 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a1cd5e53-77cc-4f7a-ad2f-b12eca4c62f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dba48c67531490685d7dd342a2661ce'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:15.944897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a1cd5e53-77cc-4f7a-ad2f-b12eca4c62f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dba48c67531490685d7dd342a2661ce'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:15.949177 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:16.001718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4c913-af7a-46df-ac5d-8434e04dfabf admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:16.036642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 815/1649] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:14:15 2022] DELETE /identity/v3/users/6dba48c67531490685d7dd342a2661ce => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:16.039585 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72727de6-501e-4db0-bf8f-825753f4fb4b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:16.040087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72727de6-501e-4db0-bf8f-825753f4fb4b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:16.040575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72727de6-501e-4db0-bf8f-825753f4fb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:16.049006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf9472-2fd7-4fad-b8e6-83ad0736e265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:16.070238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf9472-2fd7-4fad-b8e6-83ad0736e265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:16.071594 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf9472-2fd7-4fad-b8e6-83ad0736e265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:16.088326 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf9472-2fd7-4fad-b8e6-83ad0736e265 admin admin] PATH_INFO: `/v3/users/83f5c0202f7548ba9f7a35d926b0b063` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:16.102168 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf9472-2fd7-4fad-b8e6-83ad0736e265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83f5c0202f7548ba9f7a35d926b0b063)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:16.109622 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: WARNING py.warnings [None req-cabf9472-2fd7-4fad-b8e6-83ad0736e265 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:14:16.109622 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: warnings.warn(msg) Sep 06 16:14:16.109622 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: Sep 06 16:14:16.112493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf9472-2fd7-4fad-b8e6-83ad0736e265 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:16.166329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-72727de6-501e-4db0-bf8f-825753f4fb4b None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:16.187360 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-72727de6-501e-4db0-bf8f-825753f4fb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:16.519038 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cabf9472-2fd7-4fad-b8e6-83ad0736e265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83f5c0202f7548ba9f7a35d926b0b063'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:16.570959 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cabf9472-2fd7-4fad-b8e6-83ad0736e265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83f5c0202f7548ba9f7a35d926b0b063'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:16.608586 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 816/1650] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 568 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:16.653145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72727de6-501e-4db0-bf8f-825753f4fb4b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:16.656145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-72727de6-501e-4db0-bf8f-825753f4fb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:16.705039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cabf9472-2fd7-4fad-b8e6-83ad0736e265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83f5c0202f7548ba9f7a35d926b0b063'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:16.719467 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cabf9472-2fd7-4fad-b8e6-83ad0736e265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83f5c0202f7548ba9f7a35d926b0b063'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:16.759611 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cabf9472-2fd7-4fad-b8e6-83ad0736e265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83f5c0202f7548ba9f7a35d926b0b063'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:16.787310 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 835/1651] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:14:15 2022] DELETE /identity/v3/users/83f5c0202f7548ba9f7a35d926b0b063 => generated 0 bytes in 1015 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:16.931956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72727de6-501e-4db0-bf8f-825753f4fb4b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:17.163775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6c7d8-1b15-4adc-bb1d-1a204b8c7794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:17.164828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6c7d8-1b15-4adc-bb1d-1a204b8c7794 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:17.165328 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6c7d8-1b15-4adc-bb1d-1a204b8c7794 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:17.165828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6c7d8-1b15-4adc-bb1d-1a204b8c7794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:17.170939 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9ca6c7d8-1b15-4adc-bb1d-1a204b8c7794 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:17.378193 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6c7d8-1b15-4adc-bb1d-1a204b8c7794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:17.381059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6c7d8-1b15-4adc-bb1d-1a204b8c7794 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:17.400591 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:17.424788 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 817/1652] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 773 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:14:18.435606 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf9472-2fd7-4fad-b8e6-83ad0736e265 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:18.438049 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cabf9472-2fd7-4fad-b8e6-83ad0736e265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:18.467771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf9472-2fd7-4fad-b8e6-83ad0736e265 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:18.481505 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c340005-a4a8-4d27-9ca3-a696152cc4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:18.482537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c340005-a4a8-4d27-9ca3-a696152cc4c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:18.483044 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c340005-a4a8-4d27-9ca3-a696152cc4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:18.483508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c340005-a4a8-4d27-9ca3-a696152cc4c3 admin admin] PATH_INFO: `/v3/projects/347b53ac5908475abfedad60e71e9aff` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:18.485323 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c340005-a4a8-4d27-9ca3-a696152cc4c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=347b53ac5908475abfedad60e71e9aff)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:18.490142 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c340005-a4a8-4d27-9ca3-a696152cc4c3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:18.696050 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4c340005-a4a8-4d27-9ca3-a696152cc4c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '347b53ac5908475abfedad60e71e9aff', 'request_id': 'req-4c340005-a4a8-4d27-9ca3-a696152cc4c3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:18.708188 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4c340005-a4a8-4d27-9ca3-a696152cc4c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '347b53ac5908475abfedad60e71e9aff', 'request_id': 'req-4c340005-a4a8-4d27-9ca3-a696152cc4c3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:18.710250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6c7d8-1b15-4adc-bb1d-1a204b8c7794 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:18.713011 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9ca6c7d8-1b15-4adc-bb1d-1a204b8c7794 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:18.716302 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4c340005-a4a8-4d27-9ca3-a696152cc4c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '347b53ac5908475abfedad60e71e9aff', 'request_id': 'req-4c340005-a4a8-4d27-9ca3-a696152cc4c3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:18.753533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4c340005-a4a8-4d27-9ca3-a696152cc4c3 admin admin] The token cache is being invalidate because project 347b53ac5908475abfedad60e71e9aff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:14:18.754719 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4c340005-a4a8-4d27-9ca3-a696152cc4c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:18.755663 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4c340005-a4a8-4d27-9ca3-a696152cc4c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:18.761994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 836/1653] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:14:18 2022] DELETE /identity/v3/projects/347b53ac5908475abfedad60e71e9aff => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:18.769524 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7eeaee-a6b9-4c2b-9802-b2bf6ffe813a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:18.770066 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7eeaee-a6b9-4c2b-9802-b2bf6ffe813a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:18.771567 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7eeaee-a6b9-4c2b-9802-b2bf6ffe813a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:18.787005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6c7d8-1b15-4adc-bb1d-1a204b8c7794 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:18.851441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ef7eeaee-a6b9-4c2b-9802-b2bf6ffe813a None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:18.855564 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ef7eeaee-a6b9-4c2b-9802-b2bf6ffe813a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:18.923392 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e392e05-e761-4b7a-95e2-c68f2da52d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:18.925160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e392e05-e761-4b7a-95e2-c68f2da52d8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:18.925682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e392e05-e761-4b7a-95e2-c68f2da52d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:18.926174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e392e05-e761-4b7a-95e2-c68f2da52d8c admin admin] PATH_INFO: `/v3/projects/a8db5a684e1c43ff86bb2a1fc69427ab` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:18.934840 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e392e05-e761-4b7a-95e2-c68f2da52d8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8db5a684e1c43ff86bb2a1fc69427ab)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:18.944939 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: WARNING py.warnings [None req-8e392e05-e761-4b7a-95e2-c68f2da52d8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 16:14:18.944939 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: warnings.warn(msg) Sep 06 16:14:18.944939 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: Sep 06 16:14:18.949928 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e392e05-e761-4b7a-95e2-c68f2da52d8c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:18.995581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:19.074594 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 837/1654] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:19.080003 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742bd58-ba03-4505-8bc6-5dbb74b9803f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:19.081251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742bd58-ba03-4505-8bc6-5dbb74b9803f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:19.081875 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742bd58-ba03-4505-8bc6-5dbb74b9803f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:19.120239 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8e392e05-e761-4b7a-95e2-c68f2da52d8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8db5a684e1c43ff86bb2a1fc69427ab', 'request_id': 'req-8e392e05-e761-4b7a-95e2-c68f2da52d8c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:19.159992 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8e392e05-e761-4b7a-95e2-c68f2da52d8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8db5a684e1c43ff86bb2a1fc69427ab', 'request_id': 'req-8e392e05-e761-4b7a-95e2-c68f2da52d8c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:19.181114 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8e392e05-e761-4b7a-95e2-c68f2da52d8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8db5a684e1c43ff86bb2a1fc69427ab', 'request_id': 'req-8e392e05-e761-4b7a-95e2-c68f2da52d8c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:19.185256 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6742bd58-ba03-4505-8bc6-5dbb74b9803f None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:19.193003 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6742bd58-ba03-4505-8bc6-5dbb74b9803f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:19.271475 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8e392e05-e761-4b7a-95e2-c68f2da52d8c admin admin] The token cache is being invalidate because project a8db5a684e1c43ff86bb2a1fc69427ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:14:19.271832 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8e392e05-e761-4b7a-95e2-c68f2da52d8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:19.276446 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8e392e05-e761-4b7a-95e2-c68f2da52d8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:19.285137 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 818/1655] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:14:18 2022] DELETE /identity/v3/projects/a8db5a684e1c43ff86bb2a1fc69427ab => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:19.291909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c2f08-4235-48dc-8631-e2e234f2feb9 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:19.292223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c2f08-4235-48dc-8631-e2e234f2feb9 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:19.292470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c2f08-4235-48dc-8631-e2e234f2feb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:19.353063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-ee5c2f08-4235-48dc-8631-e2e234f2feb9 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:19.355117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ee5c2f08-4235-48dc-8631-e2e234f2feb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:19.441960 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 838/1656] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:14:19.446263 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742bd58-ba03-4505-8bc6-5dbb74b9803f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:19.479610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742bd58-ba03-4505-8bc6-5dbb74b9803f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:19.498130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029777d6-cf19-4ebf-b651-f33e98d2a4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:19.500295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029777d6-cf19-4ebf-b651-f33e98d2a4cf admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:19.501312 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029777d6-cf19-4ebf-b651-f33e98d2a4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:19.501812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029777d6-cf19-4ebf-b651-f33e98d2a4cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:19.507442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029777d6-cf19-4ebf-b651-f33e98d2a4cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:14:19.509133 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029777d6-cf19-4ebf-b651-f33e98d2a4cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:19.511682 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029777d6-cf19-4ebf-b651-f33e98d2a4cf admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:19.554636 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 819/1657] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:14:19.556697 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c896ac-f2d1-447c-a29d-17fd0abfe052 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:19.557385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c896ac-f2d1-447c-a29d-17fd0abfe052 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:19.557786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c896ac-f2d1-447c-a29d-17fd0abfe052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:19.558680 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 839/1658] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:14:19.562716 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029777d6-cf19-4ebf-b651-f33e98d2a4cf admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:19.622943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029777d6-cf19-4ebf-b651-f33e98d2a4cf admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:19.656654 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-a9c896ac-f2d1-447c-a29d-17fd0abfe052 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:19.662567 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a9c896ac-f2d1-447c-a29d-17fd0abfe052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:19.665116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d67bb-5b93-487f-a078-034567523487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:19.666028 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d67bb-5b93-487f-a078-034567523487 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:19.667505 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d67bb-5b93-487f-a078-034567523487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:19.667954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d67bb-5b93-487f-a078-034567523487 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:19.668765 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d67bb-5b93-487f-a078-034567523487 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:19.671369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d67bb-5b93-487f-a078-034567523487 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:19.698916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 820/1659] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 06 16:14:19.704811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c896ac-f2d1-447c-a29d-17fd0abfe052 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:19.728179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c896ac-f2d1-447c-a29d-17fd0abfe052 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:19.739404 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54387e4a-33ef-4066-9957-30c0a25062f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:19.740766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54387e4a-33ef-4066-9957-30c0a25062f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:19.746633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54387e4a-33ef-4066-9957-30c0a25062f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:19.747221 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54387e4a-33ef-4066-9957-30c0a25062f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:19.748508 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54387e4a-33ef-4066-9957-30c0a25062f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:14:19.749197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54387e4a-33ef-4066-9957-30c0a25062f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:19.764115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54387e4a-33ef-4066-9957-30c0a25062f5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:19.794897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 840/1660] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:19 2022] POST /identity/v3/projects => generated 365 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:19.797363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d67bb-5b93-487f-a078-034567523487 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:19.852037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 821/1661] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:14:19.853697 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54387e4a-33ef-4066-9957-30c0a25062f5 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:19.889587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54387e4a-33ef-4066-9957-30c0a25062f5 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:19.918405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:20.023830 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d67bb-5b93-487f-a078-034567523487 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:20.031276 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb75636-be8d-44ea-82b0-c6665df50191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:20.032277 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb75636-be8d-44ea-82b0-c6665df50191 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:20.032787 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb75636-be8d-44ea-82b0-c6665df50191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:20.033304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb75636-be8d-44ea-82b0-c6665df50191 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:20.035173 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb75636-be8d-44ea-82b0-c6665df50191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:20.037808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb75636-be8d-44ea-82b0-c6665df50191 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:20.085714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 841/1662] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:19 2022] POST /identity/v3/users => generated 283 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:14:20.087005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d0059-7bd7-4947-b791-68094a30527e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:20.088428 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d0059-7bd7-4947-b791-68094a30527e admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:20.088999 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d0059-7bd7-4947-b791-68094a30527e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:20.090293 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb75636-be8d-44ea-82b0-c6665df50191 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:20.091870 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d0059-7bd7-4947-b791-68094a30527e admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:20.095070 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d0059-7bd7-4947-b791-68094a30527e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:20.102551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d0059-7bd7-4947-b791-68094a30527e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:20.111649 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb75636-be8d-44ea-82b0-c6665df50191 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:20.138819 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad861f3-6fa8-4e91-b766-f301bbc001c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:20.140049 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad861f3-6fa8-4e91-b766-f301bbc001c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:20.140457 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad861f3-6fa8-4e91-b766-f301bbc001c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:20.141641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad861f3-6fa8-4e91-b766-f301bbc001c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:20.148615 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad861f3-6fa8-4e91-b766-f301bbc001c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:20.151706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad861f3-6fa8-4e91-b766-f301bbc001c9 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:20.195598 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 842/1663] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:20.203993 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 822/1664] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:19 2022] POST /identity/v3/projects => generated 367 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:20.210391 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d0059-7bd7-4947-b791-68094a30527e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:20.215119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad861f3-6fa8-4e91-b766-f301bbc001c9 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:20.243674 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad861f3-6fa8-4e91-b766-f301bbc001c9 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:20.378160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d0059-7bd7-4947-b791-68094a30527e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:20.385389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:20.392095 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adccd63-bfc2-4598-8cc7-e282e1167e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:20.393011 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adccd63-bfc2-4598-8cc7-e282e1167e66 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:20.397301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adccd63-bfc2-4598-8cc7-e282e1167e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:20.397440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adccd63-bfc2-4598-8cc7-e282e1167e66 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:20.398110 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adccd63-bfc2-4598-8cc7-e282e1167e66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:20.403753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adccd63-bfc2-4598-8cc7-e282e1167e66 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:20.463416 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 823/1665] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:20 2022] POST /identity/v3/users => generated 284 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:20.467125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adccd63-bfc2-4598-8cc7-e282e1167e66 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:20.498864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adccd63-bfc2-4598-8cc7-e282e1167e66 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:20.519973 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eee845-f3d7-4185-9409-a83d37fe9669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:20.520848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eee845-f3d7-4185-9409-a83d37fe9669 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:20.521133 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eee845-f3d7-4185-9409-a83d37fe9669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:20.521493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eee845-f3d7-4185-9409-a83d37fe9669 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:20.522267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eee845-f3d7-4185-9409-a83d37fe9669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:20.527769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e846cc6-0866-4e3d-bfc1-2b0d34444af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:20.528300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eee845-f3d7-4185-9409-a83d37fe9669 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:20.539359 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e846cc6-0866-4e3d-bfc1-2b0d34444af3 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:20.539359 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e846cc6-0866-4e3d-bfc1-2b0d34444af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:20.539359 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e846cc6-0866-4e3d-bfc1-2b0d34444af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:20.539359 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e846cc6-0866-4e3d-bfc1-2b0d34444af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:20.546449 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e846cc6-0866-4e3d-bfc1-2b0d34444af3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:20.556517 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 824/1666] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:20.565663 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eee845-f3d7-4185-9409-a83d37fe9669 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:20.594872 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eee845-f3d7-4185-9409-a83d37fe9669 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:20.611533 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19632d-f7b4-4648-b293-17d41c379f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:20.612735 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19632d-f7b4-4648-b293-17d41c379f55 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:20.613210 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19632d-f7b4-4648-b293-17d41c379f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:20.613564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19632d-f7b4-4648-b293-17d41c379f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:20.614318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c19632d-f7b4-4648-b293-17d41c379f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:20.619314 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c19632d-f7b4-4648-b293-17d41c379f55 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:20.619608 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 843/1667] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:20.625437 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e846cc6-0866-4e3d-bfc1-2b0d34444af3 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:20.661253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 825/1668] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:20.667729 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19632d-f7b4-4648-b293-17d41c379f55 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:20.676174 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e846cc6-0866-4e3d-bfc1-2b0d34444af3 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:20.685536 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b84d6-0a5d-4c44-8020-eb1aa0863481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:20.687002 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19632d-f7b4-4648-b293-17d41c379f55 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:20.687167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b84d6-0a5d-4c44-8020-eb1aa0863481 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:20.687577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b84d6-0a5d-4c44-8020-eb1aa0863481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:20.688271 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b84d6-0a5d-4c44-8020-eb1aa0863481 admin admin] PATH_INFO: `/v3/projects/5e1b00dc91494f03853fb224ffeef11d/users/e3c7750f19234336bf008c3474dffbf1/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:20.702433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e019ef5-ae36-4a7e-8bba-b5c671e3dcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:20.703201 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e019ef5-ae36-4a7e-8bba-b5c671e3dcc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:20.703601 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e019ef5-ae36-4a7e-8bba-b5c671e3dcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:20.703982 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e019ef5-ae36-4a7e-8bba-b5c671e3dcc7 admin admin] PATH_INFO: `/v3/projects/66f0180996ae4e07ba877a329c5b919d/users/2bcdd40ae22843ea9a3ecaf5f920c144/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:20.736182 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b84d6-0a5d-4c44-8020-eb1aa0863481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e1b00dc91494f03853fb224ffeef11d, user_id=e3c7750f19234336bf008c3474dffbf1, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:20.744485 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b84d6-0a5d-4c44-8020-eb1aa0863481 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:20.746709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e019ef5-ae36-4a7e-8bba-b5c671e3dcc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66f0180996ae4e07ba877a329c5b919d, user_id=2bcdd40ae22843ea9a3ecaf5f920c144, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:20.749727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e019ef5-ae36-4a7e-8bba-b5c671e3dcc7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:20.782724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 826/1669] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:20 2022] PUT /identity/v3/projects/66f0180996ae4e07ba877a329c5b919d/users/2bcdd40ae22843ea9a3ecaf5f920c144/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:14:20.799972 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d29b61-49b7-48c8-ac11-b739b5d12e86 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:20.801166 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d29b61-49b7-48c8-ac11-b739b5d12e86 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:20.801782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d29b61-49b7-48c8-ac11-b739b5d12e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:20.838905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 844/1670] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:20 2022] PUT /identity/v3/projects/5e1b00dc91494f03853fb224ffeef11d/users/e3c7750f19234336bf008c3474dffbf1/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:20.854077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9531a69-77dc-43f4-9122-b201bfe9fc61 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:20.854592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9531a69-77dc-43f4-9122-b201bfe9fc61 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:20.854789 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9531a69-77dc-43f4-9122-b201bfe9fc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:20.869005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-c4d29b61-49b7-48c8-ac11-b739b5d12e86 None None] MFA Rules not processed for user `2bcdd40ae22843ea9a3ecaf5f920c144`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:20.870499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c4d29b61-49b7-48c8-ac11-b739b5d12e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:20.946842 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d9531a69-77dc-43f4-9122-b201bfe9fc61 None None] MFA Rules not processed for user `e3c7750f19234336bf008c3474dffbf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:20.948223 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d9531a69-77dc-43f4-9122-b201bfe9fc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:20.978096 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:21.141059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 827/1671] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:14:21.147660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d29b61-49b7-48c8-ac11-b739b5d12e86 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:21.156699 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 845/1672] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:21.168511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9531a69-77dc-43f4-9122-b201bfe9fc61 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:21.418462 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d29b61-49b7-48c8-ac11-b739b5d12e86 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:21.425992 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9531a69-77dc-43f4-9122-b201bfe9fc61 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:21.433883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c51cb-39f7-4903-87ee-a874f599d8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:21.434945 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c51cb-39f7-4903-87ee-a874f599d8b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:21.435325 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c51cb-39f7-4903-87ee-a874f599d8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:21.435605 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c51cb-39f7-4903-87ee-a874f599d8b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:21.435993 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722e243-4fe0-4195-806b-e1f23cc34216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:21.436633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c51cb-39f7-4903-87ee-a874f599d8b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:21.437263 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722e243-4fe0-4195-806b-e1f23cc34216 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:21.437840 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722e243-4fe0-4195-806b-e1f23cc34216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:21.438275 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722e243-4fe0-4195-806b-e1f23cc34216 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:21.439070 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c51cb-39f7-4903-87ee-a874f599d8b2 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:21.442140 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722e243-4fe0-4195-806b-e1f23cc34216 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:21.450280 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722e243-4fe0-4195-806b-e1f23cc34216 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:21.503158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 846/1673] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:21 2022] POST /identity/v3/projects => generated 365 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:21.505463 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184d29c-fed9-46fe-95ac-8fcf63bc54af None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:21.506371 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184d29c-fed9-46fe-95ac-8fcf63bc54af None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:21.506851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184d29c-fed9-46fe-95ac-8fcf63bc54af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:21.568405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 828/1674] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:21 2022] POST /identity/v3/projects => generated 369 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:14:21.570278 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722e243-4fe0-4195-806b-e1f23cc34216 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:21.589367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7184d29c-fed9-46fe-95ac-8fcf63bc54af None None] MFA Rules not processed for user `eda6839efaab4317a0eda1f47f1656a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:21.591585 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7184d29c-fed9-46fe-95ac-8fcf63bc54af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:21.622003 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:21.809890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722e243-4fe0-4195-806b-e1f23cc34216 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:21.850837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 847/1675] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:21.863943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184d29c-fed9-46fe-95ac-8fcf63bc54af None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:21.865310 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6f5f6-c062-4434-a764-09608131c7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:21.870977 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6f5f6-c062-4434-a764-09608131c7e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:21.872883 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6f5f6-c062-4434-a764-09608131c7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:21.874387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6f5f6-c062-4434-a764-09608131c7e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:21.875514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6f5f6-c062-4434-a764-09608131c7e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:21.888198 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6f5f6-c062-4434-a764-09608131c7e9 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:21.915970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184d29c-fed9-46fe-95ac-8fcf63bc54af None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:21.935770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0e038-9393-4981-8be3-25ba22b4688e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:21.937127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0e038-9393-4981-8be3-25ba22b4688e admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:21.937622 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0e038-9393-4981-8be3-25ba22b4688e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:21.938017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0e038-9393-4981-8be3-25ba22b4688e admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:21.943270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0e038-9393-4981-8be3-25ba22b4688e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:21.947185 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0e038-9393-4981-8be3-25ba22b4688e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:21.957266 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 829/1676] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:21 2022] POST /identity/v3/users => generated 289 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:21.957266 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c3481-cb72-4157-95b1-2fb7be027722 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:21.957266 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c3481-cb72-4157-95b1-2fb7be027722 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:21.957266 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c3481-cb72-4157-95b1-2fb7be027722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:22.008873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 848/1677] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:21 2022] POST /identity/v3/users => generated 291 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:22.010402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0e038-9393-4981-8be3-25ba22b4688e admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:22.012822 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d68c3481-cb72-4157-95b1-2fb7be027722 None None] MFA Rules not processed for user `eda6839efaab4317a0eda1f47f1656a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:22.014861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d68c3481-cb72-4157-95b1-2fb7be027722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:22.040578 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0e038-9393-4981-8be3-25ba22b4688e admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:22.048183 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 830/1678] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:22.052208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c3481-cb72-4157-95b1-2fb7be027722 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:22.062438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4131a53-ceff-448f-9df7-2bb6dec8223e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:22.071052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4131a53-ceff-448f-9df7-2bb6dec8223e admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:22.071428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4131a53-ceff-448f-9df7-2bb6dec8223e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:22.071705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4131a53-ceff-448f-9df7-2bb6dec8223e admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:22.072418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4131a53-ceff-448f-9df7-2bb6dec8223e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:22.074133 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4131a53-ceff-448f-9df7-2bb6dec8223e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:22.084041 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c3481-cb72-4157-95b1-2fb7be027722 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:22.102091 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 849/1679] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:22.104979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf12a0-039f-4bec-b81e-47bc78e0c875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:22.105639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf12a0-039f-4bec-b81e-47bc78e0c875 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:22.106526 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf12a0-039f-4bec-b81e-47bc78e0c875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:22.106526 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf12a0-039f-4bec-b81e-47bc78e0c875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:22.107064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf12a0-039f-4bec-b81e-47bc78e0c875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:22.108824 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf12a0-039f-4bec-b81e-47bc78e0c875 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:22.111707 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4131a53-ceff-448f-9df7-2bb6dec8223e admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:22.113656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b4131a53-ceff-448f-9df7-2bb6dec8223e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:22.131504 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 831/1680] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:22.132490 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf12a0-039f-4bec-b81e-47bc78e0c875 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:22.166732 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf12a0-039f-4bec-b81e-47bc78e0c875 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:22.196524 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad82b35-3294-4bcd-aeed-01c7da3a8091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:22.197703 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad82b35-3294-4bcd-aeed-01c7da3a8091 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:22.198191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad82b35-3294-4bcd-aeed-01c7da3a8091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:22.199088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad82b35-3294-4bcd-aeed-01c7da3a8091 admin admin] PATH_INFO: `/v3/projects/4d112432449d40278e0a873ec14bd54f/users/297ae31f4e324b6fb5940d799cfae5fc/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:22.290400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad82b35-3294-4bcd-aeed-01c7da3a8091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d112432449d40278e0a873ec14bd54f, user_id=297ae31f4e324b6fb5940d799cfae5fc, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:22.308022 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad82b35-3294-4bcd-aeed-01c7da3a8091 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:22.351426 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:22.360531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 832/1681] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:22 2022] PUT /identity/v3/projects/4d112432449d40278e0a873ec14bd54f/users/297ae31f4e324b6fb5940d799cfae5fc/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:22.360531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad82b35-3294-4bcd-aeed-01c7da3a8091 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:22.448913 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4131a53-ceff-448f-9df7-2bb6dec8223e admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:22.559842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad82b35-3294-4bcd-aeed-01c7da3a8091 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:22.587085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac994068-b126-4c51-ba82-7f7dfe3ffc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:22.587896 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac994068-b126-4c51-ba82-7f7dfe3ffc8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:22.589819 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac994068-b126-4c51-ba82-7f7dfe3ffc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:22.589819 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac994068-b126-4c51-ba82-7f7dfe3ffc8b admin admin] PATH_INFO: `/v3/projects/7bc923f0767745309a0b4471ff64bae0/users/1eb8443f91134749aa737ddcdf09de9f/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:22.624616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152dd31-bd47-4774-8baa-91c2e72443aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:22.625261 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152dd31-bd47-4774-8baa-91c2e72443aa service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:22.625546 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152dd31-bd47-4774-8baa-91c2e72443aa service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:22.625821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152dd31-bd47-4774-8baa-91c2e72443aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:22.641375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac994068-b126-4c51-ba82-7f7dfe3ffc8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bc923f0767745309a0b4471ff64bae0, user_id=1eb8443f91134749aa737ddcdf09de9f, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:22.645039 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac994068-b126-4c51-ba82-7f7dfe3ffc8b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:22.664024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:22.671412 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 833/1682] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:22 2022] PUT /identity/v3/projects/7bc923f0767745309a0b4471ff64bae0/users/1eb8443f91134749aa737ddcdf09de9f/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:22.672846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac994068-b126-4c51-ba82-7f7dfe3ffc8b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:22.871856 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac994068-b126-4c51-ba82-7f7dfe3ffc8b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:22.888881 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152dd31-bd47-4774-8baa-91c2e72443aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:22.891993 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152dd31-bd47-4774-8baa-91c2e72443aa service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:22.895156 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a9234-2de5-4100-a848-5152dff181b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:22.895881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a9234-2de5-4100-a848-5152dff181b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:22.896189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a9234-2de5-4100-a848-5152dff181b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:22.896489 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a9234-2de5-4100-a848-5152dff181b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:22.897220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a9234-2de5-4100-a848-5152dff181b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:22.904911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a9234-2de5-4100-a848-5152dff181b2 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:22.935329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 850/1683] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:22 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 833 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:22.947607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 834/1684] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:22.957142 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a9234-2de5-4100-a848-5152dff181b2 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:22.957550 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152dd31-bd47-4774-8baa-91c2e72443aa service nova] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:22.994989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a9234-2de5-4100-a848-5152dff181b2 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:23.001781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152dd31-bd47-4774-8baa-91c2e72443aa service nova] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:23.019235 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced88761-dd46-4bbb-8e06-2eb4e778f167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:23.020358 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc9335-a7dd-4e2e-be0d-513c41b78d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:23.021249 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc9335-a7dd-4e2e-be0d-513c41b78d97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:23.021459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc9335-a7dd-4e2e-be0d-513c41b78d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:23.021769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc9335-a7dd-4e2e-be0d-513c41b78d97 admin admin] PATH_INFO: `/v3/projects/4d112432449d40278e0a873ec14bd54f/users/297ae31f4e324b6fb5940d799cfae5fc/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:23.023300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced88761-dd46-4bbb-8e06-2eb4e778f167 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:23.023876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced88761-dd46-4bbb-8e06-2eb4e778f167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:23.024362 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced88761-dd46-4bbb-8e06-2eb4e778f167 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:23.025996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced88761-dd46-4bbb-8e06-2eb4e778f167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:23.027849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc9335-a7dd-4e2e-be0d-513c41b78d97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d112432449d40278e0a873ec14bd54f, user_id=297ae31f4e324b6fb5940d799cfae5fc, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:23.029206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced88761-dd46-4bbb-8e06-2eb4e778f167 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:23.038555 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc9335-a7dd-4e2e-be0d-513c41b78d97 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:23.056679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 851/1685] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:23.063442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced88761-dd46-4bbb-8e06-2eb4e778f167 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:23.096975 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 835/1686] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:22 2022] PUT /identity/v3/projects/4d112432449d40278e0a873ec14bd54f/users/297ae31f4e324b6fb5940d799cfae5fc/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:23.111378 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf4fab-2e5d-4145-8875-55c9e7253d5c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:23.120612 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf4fab-2e5d-4145-8875-55c9e7253d5c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:23.121268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf4fab-2e5d-4145-8875-55c9e7253d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:23.125633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced88761-dd46-4bbb-8e06-2eb4e778f167 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:23.285915 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1fcf4fab-2e5d-4145-8875-55c9e7253d5c None None] MFA Rules not processed for user `297ae31f4e324b6fb5940d799cfae5fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:23.293404 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1fcf4fab-2e5d-4145-8875-55c9e7253d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:23.477574 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdd4a8-5ec0-404b-8cac-fee4b177d8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:23.478274 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdd4a8-5ec0-404b-8cac-fee4b177d8fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:23.478561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdd4a8-5ec0-404b-8cac-fee4b177d8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:23.478876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdd4a8-5ec0-404b-8cac-fee4b177d8fe admin admin] PATH_INFO: `/v3/projects/7bc923f0767745309a0b4471ff64bae0/users/1eb8443f91134749aa737ddcdf09de9f/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:23.483674 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bdd4a8-5ec0-404b-8cac-fee4b177d8fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bc923f0767745309a0b4471ff64bae0, user_id=1eb8443f91134749aa737ddcdf09de9f, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:23.491996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bdd4a8-5ec0-404b-8cac-fee4b177d8fe admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:23.506594 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:23.535862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 852/1687] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:23 2022] PUT /identity/v3/projects/7bc923f0767745309a0b4471ff64bae0/users/1eb8443f91134749aa737ddcdf09de9f/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:23.537388 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3de9a8-0b9f-46cb-9fd9-e2417c231821 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:23.537681 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3de9a8-0b9f-46cb-9fd9-e2417c231821 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:23.537960 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3de9a8-0b9f-46cb-9fd9-e2417c231821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:23.582842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 836/1688] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:23.604799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-1f3de9a8-0b9f-46cb-9fd9-e2417c231821 None None] MFA Rules not processed for user `1eb8443f91134749aa737ddcdf09de9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:23.606652 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1f3de9a8-0b9f-46cb-9fd9-e2417c231821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:23.607780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf4fab-2e5d-4145-8875-55c9e7253d5c None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:23.610296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1fcf4fab-2e5d-4145-8875-55c9e7253d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:23.762356 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf4fab-2e5d-4145-8875-55c9e7253d5c None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:23.880375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195bfba-27e1-46a7-9cb8-a468fed50ce3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:23.881041 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195bfba-27e1-46a7-9cb8-a468fed50ce3 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:23.881301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195bfba-27e1-46a7-9cb8-a468fed50ce3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:23.881581 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195bfba-27e1-46a7-9cb8-a468fed50ce3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:24.070860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 853/1689] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 534 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:24.079473 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3de9a8-0b9f-46cb-9fd9-e2417c231821 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:24.082616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1f3de9a8-0b9f-46cb-9fd9-e2417c231821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:24.115202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3de9a8-0b9f-46cb-9fd9-e2417c231821 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:24.125171 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e181c-9932-4b90-95cf-c5bee5f320d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:24.127168 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e181c-9932-4b90-95cf-c5bee5f320d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:24.128432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e181c-9932-4b90-95cf-c5bee5f320d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:24.130384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195bfba-27e1-46a7-9cb8-a468fed50ce3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:24.130802 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e181c-9932-4b90-95cf-c5bee5f320d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:24.141375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195bfba-27e1-46a7-9cb8-a468fed50ce3 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:24.161446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e181c-9932-4b90-95cf-c5bee5f320d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:24.165498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e181c-9932-4b90-95cf-c5bee5f320d3 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:24.172665 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 837/1690] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:23 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:24.184887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 854/1691] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:24.239977 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32caacf4-e1ff-400c-afe8-7f951cbdd043 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:24.242424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32caacf4-e1ff-400c-afe8-7f951cbdd043 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:24.242797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32caacf4-e1ff-400c-afe8-7f951cbdd043 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:24.248888 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 838/1692] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 6 16:14:24 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:14:24.267182 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefaf37-a30e-4805-8c4f-90de1d954766 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:24.267736 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefaf37-a30e-4805-8c4f-90de1d954766 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:24.268258 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefaf37-a30e-4805-8c4f-90de1d954766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:24.351251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:24.435025 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8fefaf37-a30e-4805-8c4f-90de1d954766 None None] MFA Rules not processed for user `989beaca146d4f9b950f333bd3567e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:24.438025 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8fefaf37-a30e-4805-8c4f-90de1d954766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:24.568602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efd2ed-de9a-40bc-894e-41c983888834 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:24.568914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efd2ed-de9a-40bc-894e-41c983888834 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:24.569164 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efd2ed-de9a-40bc-894e-41c983888834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:24.655000 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 855/1693] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 6 16:14:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:24.699475 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b1efd2ed-de9a-40bc-894e-41c983888834 None None] MFA Rules not processed for user `668452994ad942069d30f4bddbba15b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:24.705047 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b1efd2ed-de9a-40bc-894e-41c983888834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:24.714024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefaf37-a30e-4805-8c4f-90de1d954766 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:24.715719 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8fefaf37-a30e-4805-8c4f-90de1d954766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:24.741791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefaf37-a30e-4805-8c4f-90de1d954766 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:24.764689 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dc142-9d4a-445f-a00c-1b2b73d5d759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:24.765330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dc142-9d4a-445f-a00c-1b2b73d5d759 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:24.765614 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dc142-9d4a-445f-a00c-1b2b73d5d759 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:24.765895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dc142-9d4a-445f-a00c-1b2b73d5d759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:24.791159 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445dc142-9d4a-445f-a00c-1b2b73d5d759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:24.792819 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445dc142-9d4a-445f-a00c-1b2b73d5d759 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:24.816273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 856/1694] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:24.818389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dc142-9d4a-445f-a00c-1b2b73d5d759 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:24.833525 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dc142-9d4a-445f-a00c-1b2b73d5d759 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:24.839137 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f847d-7fcd-4d18-970a-4d0b867504c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:24.840010 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f847d-7fcd-4d18-970a-4d0b867504c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:24.840368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f847d-7fcd-4d18-970a-4d0b867504c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:24.840743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f847d-7fcd-4d18-970a-4d0b867504c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:24.892203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:24.948359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 839/1695] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 06 16:14:24.959659 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2a024-f6ed-4f3e-96c2-3f847f612977 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:24.959933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2a024-f6ed-4f3e-96c2-3f847f612977 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:24.960212 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2a024-f6ed-4f3e-96c2-3f847f612977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:24.995367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-42d2a024-f6ed-4f3e-96c2-3f847f612977 None None] MFA Rules not processed for user `668452994ad942069d30f4bddbba15b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:24.996112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5f847d-7fcd-4d18-970a-4d0b867504c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:24.997271 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-42d2a024-f6ed-4f3e-96c2-3f847f612977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:24.998045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5f847d-7fcd-4d18-970a-4d0b867504c6 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:25.012733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 840/1696] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:14:25.019490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 857/1697] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:25.304198 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2a024-f6ed-4f3e-96c2-3f847f612977 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:25.305827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-42d2a024-f6ed-4f3e-96c2-3f847f612977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:25.462752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f502a0-a7b0-44c0-8ee6-3c9d302433f7 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:25.463642 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f502a0-a7b0-44c0-8ee6-3c9d302433f7 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:25.463925 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f502a0-a7b0-44c0-8ee6-3c9d302433f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:25.513707 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-39f502a0-a7b0-44c0-8ee6-3c9d302433f7 None None] MFA Rules not processed for user `95ac6a289fb345d3a98871f5e11f9414`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:25.515074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-39f502a0-a7b0-44c0-8ee6-3c9d302433f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:25.534878 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2a024-f6ed-4f3e-96c2-3f847f612977 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:25.555333 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bf21f-0425-4ee6-8cda-121481e96dbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:25.555823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bf21f-0425-4ee6-8cda-121481e96dbc service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:25.556278 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bf21f-0425-4ee6-8cda-121481e96dbc service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:25.557071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bf21f-0425-4ee6-8cda-121481e96dbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:25.576674 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bf21f-0425-4ee6-8cda-121481e96dbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:25.579043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bf21f-0425-4ee6-8cda-121481e96dbc service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:25.622265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 841/1698] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:14:25.666222 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 858/1699] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:14:26.312283 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bf21f-0425-4ee6-8cda-121481e96dbc service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:26.315218 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bb3bf21f-0425-4ee6-8cda-121481e96dbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:26.322341 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f502a0-a7b0-44c0-8ee6-3c9d302433f7 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:26.366103 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f502a0-a7b0-44c0-8ee6-3c9d302433f7 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:26.430951 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b76063-b785-45d5-a337-6f65c5bedf68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:26.432308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b76063-b785-45d5-a337-6f65c5bedf68 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:26.438689 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b76063-b785-45d5-a337-6f65c5bedf68 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:26.441042 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b76063-b785-45d5-a337-6f65c5bedf68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:26.473756 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b76063-b785-45d5-a337-6f65c5bedf68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:26.478410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b76063-b785-45d5-a337-6f65c5bedf68 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:26.522376 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 859/1700] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:26.611884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bf21f-0425-4ee6-8cda-121481e96dbc service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:26.626403 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2b053-189e-4e8c-86df-04d7d59f94e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034d75ab747e4c0cab040e47c6a6236e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:26.629520 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2b053-189e-4e8c-86df-04d7d59f94e2 service placement] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:26.630051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2b053-189e-4e8c-86df-04d7d59f94e2 service placement] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:26.630699 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2b053-189e-4e8c-86df-04d7d59f94e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:26.634326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a7f2b053-189e-4e8c-86df-04d7d59f94e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:26.710345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2b053-189e-4e8c-86df-04d7d59f94e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:26.712214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2b053-189e-4e8c-86df-04d7d59f94e2 service placement] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:26.758850 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 842/1701] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:14:26.869909 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f93bd-da72-40c8-9862-7bb673fec870 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:26.870816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f93bd-da72-40c8-9862-7bb673fec870 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:26.871482 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f93bd-da72-40c8-9862-7bb673fec870 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:26.875505 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 860/1702] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 6 16:14:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:14:26.891100 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da72761-0231-4737-b184-0284290a360a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:26.892121 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da72761-0231-4737-b184-0284290a360a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:26.893959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da72761-0231-4737-b184-0284290a360a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:26.963628 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-7da72761-0231-4737-b184-0284290a360a None None] MFA Rules not processed for user `989beaca146d4f9b950f333bd3567e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:26.964916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7da72761-0231-4737-b184-0284290a360a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:26.990794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 843/1703] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 6 16:14:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:27.004771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f93bd-da72-40c8-9862-7bb673fec870 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:27.005659 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2f9f93bd-da72-40c8-9862-7bb673fec870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:27.032877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f93bd-da72-40c8-9862-7bb673fec870 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:27.044922 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e537e93-b0e3-4eaa-bb41-88b7f5010a82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:27.046047 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e537e93-b0e3-4eaa-bb41-88b7f5010a82 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:27.046492 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e537e93-b0e3-4eaa-bb41-88b7f5010a82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:27.047031 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e537e93-b0e3-4eaa-bb41-88b7f5010a82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:27.065948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:27.069446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e537e93-b0e3-4eaa-bb41-88b7f5010a82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:27.071440 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e537e93-b0e3-4eaa-bb41-88b7f5010a82 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:27.107394 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 861/1704] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:27.312382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd653d-ec02-414c-a0da-a54b95877561 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:27.313304 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd653d-ec02-414c-a0da-a54b95877561 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:27.313784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd653d-ec02-414c-a0da-a54b95877561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:27.367763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-82cd653d-ec02-414c-a0da-a54b95877561 None None] MFA Rules not processed for user `eda6839efaab4317a0eda1f47f1656a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:27.378289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-82cd653d-ec02-414c-a0da-a54b95877561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:27.656733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:27.741129 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 844/1705] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:14:30.259520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e537e93-b0e3-4eaa-bb41-88b7f5010a82 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:30.557043 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e537e93-b0e3-4eaa-bb41-88b7f5010a82 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:30.573447 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d20d0-5661-4b02-9553-8cdd774c74a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:30.574400 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d20d0-5661-4b02-9553-8cdd774c74a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:30.574723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d20d0-5661-4b02-9553-8cdd774c74a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:30.575004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d20d0-5661-4b02-9553-8cdd774c74a6 admin admin] PATH_INFO: `/v3/users/2bcdd40ae22843ea9a3ecaf5f920c144` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:30.576917 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d20d0-5661-4b02-9553-8cdd774c74a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bcdd40ae22843ea9a3ecaf5f920c144)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:30.582557 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d20d0-5661-4b02-9553-8cdd774c74a6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:30.730836 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a02d20d0-5661-4b02-9553-8cdd774c74a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bcdd40ae22843ea9a3ecaf5f920c144'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:30.752487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a02d20d0-5661-4b02-9553-8cdd774c74a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bcdd40ae22843ea9a3ecaf5f920c144'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:30.866781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a02d20d0-5661-4b02-9553-8cdd774c74a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bcdd40ae22843ea9a3ecaf5f920c144'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:30.875996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a02d20d0-5661-4b02-9553-8cdd774c74a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bcdd40ae22843ea9a3ecaf5f920c144'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:30.913734 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a02d20d0-5661-4b02-9553-8cdd774c74a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bcdd40ae22843ea9a3ecaf5f920c144'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:30.927618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 862/1706] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:14:30 2022] DELETE /identity/v3/users/2bcdd40ae22843ea9a3ecaf5f920c144 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:30.933687 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e8449-ce83-45bd-8267-8010b641af6d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:30.934468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e8449-ce83-45bd-8267-8010b641af6d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:30.934468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e8449-ce83-45bd-8267-8010b641af6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:30.994822 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-154e8449-ce83-45bd-8267-8010b641af6d None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:31.007055 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-154e8449-ce83-45bd-8267-8010b641af6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:31.149725 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 845/1707] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:31.202190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d20d0-5661-4b02-9553-8cdd774c74a6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:31.204205 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a02d20d0-5661-4b02-9553-8cdd774c74a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:31.349869 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e8449-ce83-45bd-8267-8010b641af6d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:31.352191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-154e8449-ce83-45bd-8267-8010b641af6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:31.396507 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e8449-ce83-45bd-8267-8010b641af6d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:31.430509 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184e4a1-be56-4854-aa26-37205839f462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:31.432029 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184e4a1-be56-4854-aa26-37205839f462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:31.432674 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184e4a1-be56-4854-aa26-37205839f462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:31.433159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184e4a1-be56-4854-aa26-37205839f462 admin admin] PATH_INFO: `/v3/users/c7e629d1f7ec43d8af4fbff93b6bffad` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:31.435328 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184e4a1-be56-4854-aa26-37205839f462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7e629d1f7ec43d8af4fbff93b6bffad)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:31.437994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184e4a1-be56-4854-aa26-37205839f462 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:31.537644 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d20d0-5661-4b02-9553-8cdd774c74a6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:31.572281 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b851618-9bfa-4f61-8125-9953227de0c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:31.579527 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b851618-9bfa-4f61-8125-9953227de0c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:31.579851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b851618-9bfa-4f61-8125-9953227de0c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:31.581873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b851618-9bfa-4f61-8125-9953227de0c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:31.594636 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f184e4a1-be56-4854-aa26-37205839f462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7e629d1f7ec43d8af4fbff93b6bffad'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:31.640242 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f184e4a1-be56-4854-aa26-37205839f462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7e629d1f7ec43d8af4fbff93b6bffad'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:31.642526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b851618-9bfa-4f61-8125-9953227de0c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:31.654225 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b851618-9bfa-4f61-8125-9953227de0c0 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:31.714920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f184e4a1-be56-4854-aa26-37205839f462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7e629d1f7ec43d8af4fbff93b6bffad'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:31.720610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 863/1708] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:31.721607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f184e4a1-be56-4854-aa26-37205839f462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7e629d1f7ec43d8af4fbff93b6bffad'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:31.724674 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f184e4a1-be56-4854-aa26-37205839f462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7e629d1f7ec43d8af4fbff93b6bffad'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:31.784781 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 846/1709] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:14:31 2022] DELETE /identity/v3/users/c7e629d1f7ec43d8af4fbff93b6bffad => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:14:31.790904 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305cfc8-bb9b-4014-9f75-1224f57c2202 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:31.791188 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305cfc8-bb9b-4014-9f75-1224f57c2202 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:31.791679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305cfc8-bb9b-4014-9f75-1224f57c2202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:31.829750 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8305cfc8-bb9b-4014-9f75-1224f57c2202 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:31.832158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8305cfc8-bb9b-4014-9f75-1224f57c2202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:31.863543 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:32.153345 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 864/1710] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:32.257140 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184e4a1-be56-4854-aa26-37205839f462 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:32.260220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f184e4a1-be56-4854-aa26-37205839f462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:32.541204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184e4a1-be56-4854-aa26-37205839f462 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:32.576119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba494d3-0754-43bf-af3f-5baf4ac2a1dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:32.577452 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba494d3-0754-43bf-af3f-5baf4ac2a1dc service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:32.578064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba494d3-0754-43bf-af3f-5baf4ac2a1dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:32.578629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba494d3-0754-43bf-af3f-5baf4ac2a1dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:32.657584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba494d3-0754-43bf-af3f-5baf4ac2a1dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:32.657584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba494d3-0754-43bf-af3f-5baf4ac2a1dc service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:32.693208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 847/1711] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:33.685994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753196d-7e5a-44b5-b51c-19a4497d88af None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:33.687058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753196d-7e5a-44b5-b51c-19a4497d88af None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:33.687058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753196d-7e5a-44b5-b51c-19a4497d88af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:33.781818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba494d3-0754-43bf-af3f-5baf4ac2a1dc service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:33.790265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8ba494d3-0754-43bf-af3f-5baf4ac2a1dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:33.840566 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba494d3-0754-43bf-af3f-5baf4ac2a1dc service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:33.855277 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815387c-f173-47ae-b3b0-79748fc1c67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:33.856332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815387c-f173-47ae-b3b0-79748fc1c67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:33.857888 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815387c-f173-47ae-b3b0-79748fc1c67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:33.858427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815387c-f173-47ae-b3b0-79748fc1c67a admin admin] PATH_INFO: `/v3/projects/66f0180996ae4e07ba877a329c5b919d` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:33.860112 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815387c-f173-47ae-b3b0-79748fc1c67a admin admin] RBAC: Authorizing `identity:delete_project(project_id=66f0180996ae4e07ba877a329c5b919d)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:33.863811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815387c-f173-47ae-b3b0-79748fc1c67a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:33.867732 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:33.890515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-9753196d-7e5a-44b5-b51c-19a4497d88af None None] MFA Rules not processed for user `ba58a4914ddf418db9e8b9dce5784cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:33.892604 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9753196d-7e5a-44b5-b51c-19a4497d88af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:34.167515 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0815387c-f173-47ae-b3b0-79748fc1c67a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66f0180996ae4e07ba877a329c5b919d', 'request_id': 'req-0815387c-f173-47ae-b3b0-79748fc1c67a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:34.183709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0815387c-f173-47ae-b3b0-79748fc1c67a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66f0180996ae4e07ba877a329c5b919d', 'request_id': 'req-0815387c-f173-47ae-b3b0-79748fc1c67a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:34.205022 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0815387c-f173-47ae-b3b0-79748fc1c67a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66f0180996ae4e07ba877a329c5b919d', 'request_id': 'req-0815387c-f173-47ae-b3b0-79748fc1c67a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:34.228406 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 865/1712] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 6 16:14:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 543 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:14:34.303287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0815387c-f173-47ae-b3b0-79748fc1c67a admin admin] The token cache is being invalidate because project 66f0180996ae4e07ba877a329c5b919d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:14:34.306284 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0815387c-f173-47ae-b3b0-79748fc1c67a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:34.309728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0815387c-f173-47ae-b3b0-79748fc1c67a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:34.318347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 848/1713] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:14:33 2022] DELETE /identity/v3/projects/66f0180996ae4e07ba877a329c5b919d => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:34.327272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753196d-7e5a-44b5-b51c-19a4497d88af None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:34.405822 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753196d-7e5a-44b5-b51c-19a4497d88af None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:34.429698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815387c-f173-47ae-b3b0-79748fc1c67a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:34.431572 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0815387c-f173-47ae-b3b0-79748fc1c67a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:34.592357 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a12de98-b252-443c-81bf-cb8447814f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:34.593936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12de98-b252-443c-81bf-cb8447814f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:34.594812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12de98-b252-443c-81bf-cb8447814f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:34.595536 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12de98-b252-443c-81bf-cb8447814f13 admin admin] PATH_INFO: `/v3/users/1eb8443f91134749aa737ddcdf09de9f` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:34.621004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a12de98-b252-443c-81bf-cb8447814f13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eb8443f91134749aa737ddcdf09de9f)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:34.626231 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a12de98-b252-443c-81bf-cb8447814f13 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:34.669864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815387c-f173-47ae-b3b0-79748fc1c67a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:34.686945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e07681-e5f7-4635-a2e8-b268bc3b4b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:34.687618 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e07681-e5f7-4635-a2e8-b268bc3b4b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:34.687891 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e07681-e5f7-4635-a2e8-b268bc3b4b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:34.688989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e07681-e5f7-4635-a2e8-b268bc3b4b57 admin admin] PATH_INFO: `/v3/projects/43dcc617cde94adf87bfa665e6cfa39c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:34.691792 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e07681-e5f7-4635-a2e8-b268bc3b4b57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43dcc617cde94adf87bfa665e6cfa39c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:34.694201 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e07681-e5f7-4635-a2e8-b268bc3b4b57 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:34.760507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:34.791088 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9a12de98-b252-443c-81bf-cb8447814f13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eb8443f91134749aa737ddcdf09de9f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:34.803582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9a12de98-b252-443c-81bf-cb8447814f13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eb8443f91134749aa737ddcdf09de9f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:34.832022 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9a12de98-b252-443c-81bf-cb8447814f13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eb8443f91134749aa737ddcdf09de9f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:34.833849 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9a12de98-b252-443c-81bf-cb8447814f13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eb8443f91134749aa737ddcdf09de9f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:34.851115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-50e07681-e5f7-4635-a2e8-b268bc3b4b57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43dcc617cde94adf87bfa665e6cfa39c', 'request_id': 'req-50e07681-e5f7-4635-a2e8-b268bc3b4b57'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:34.852854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9a12de98-b252-443c-81bf-cb8447814f13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eb8443f91134749aa737ddcdf09de9f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:34.869769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 866/1714] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:14:34 2022] DELETE /identity/v3/users/1eb8443f91134749aa737ddcdf09de9f => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:34.871632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a12de98-b252-443c-81bf-cb8447814f13 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:34.874751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9a12de98-b252-443c-81bf-cb8447814f13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:34.884967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-50e07681-e5f7-4635-a2e8-b268bc3b4b57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43dcc617cde94adf87bfa665e6cfa39c', 'request_id': 'req-50e07681-e5f7-4635-a2e8-b268bc3b4b57'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:34.901608 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-50e07681-e5f7-4635-a2e8-b268bc3b4b57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43dcc617cde94adf87bfa665e6cfa39c', 'request_id': 'req-50e07681-e5f7-4635-a2e8-b268bc3b4b57'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:34.930234 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-50e07681-e5f7-4635-a2e8-b268bc3b4b57 admin admin] The token cache is being invalidate because project 43dcc617cde94adf87bfa665e6cfa39c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:14:34.931193 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-50e07681-e5f7-4635-a2e8-b268bc3b4b57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:34.933343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-50e07681-e5f7-4635-a2e8-b268bc3b4b57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:34.936814 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 849/1715] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:14:34 2022] DELETE /identity/v3/projects/43dcc617cde94adf87bfa665e6cfa39c => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:14:34.947782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e0e23-f13b-4d56-89c8-70cae664a320 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:34.951476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e0e23-f13b-4d56-89c8-70cae664a320 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:34.952375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e0e23-f13b-4d56-89c8-70cae664a320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:35.004340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-4a0e0e23-f13b-4d56-89c8-70cae664a320 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:35.007342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4a0e0e23-f13b-4d56-89c8-70cae664a320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:35.094585 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a12de98-b252-443c-81bf-cb8447814f13 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:35.099746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:35.103273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043702d-ef99-4f2b-937c-f23c02dd90e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ece176d412574790a13d03dcca08f91a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:35.104260 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043702d-ef99-4f2b-937c-f23c02dd90e8 service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:35.104714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043702d-ef99-4f2b-937c-f23c02dd90e8 service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:35.105054 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043702d-ef99-4f2b-937c-f23c02dd90e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:35.108239 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0043702d-ef99-4f2b-937c-f23c02dd90e8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:35.286087 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043702d-ef99-4f2b-937c-f23c02dd90e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:35.289310 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043702d-ef99-4f2b-937c-f23c02dd90e8 service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:35.307730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 850/1716] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:35.315712 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 867/1717] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 6 16:14:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:14:35.322759 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2131e-7aa3-4437-b74a-38259c9f2ba1 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:35.323427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2131e-7aa3-4437-b74a-38259c9f2ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:35.324257 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2131e-7aa3-4437-b74a-38259c9f2ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:35.368986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-6ad2131e-7aa3-4437-b74a-38259c9f2ba1 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:35.372220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6ad2131e-7aa3-4437-b74a-38259c9f2ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:35.394590 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 851/1718] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:35.401825 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043702d-ef99-4f2b-937c-f23c02dd90e8 service swift] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:35.475023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043702d-ef99-4f2b-937c-f23c02dd90e8 service swift] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:35.521231 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0126a7a-8173-487b-b2a3-8545125ca8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:35.537384 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0126a7a-8173-487b-b2a3-8545125ca8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:35.537384 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0126a7a-8173-487b-b2a3-8545125ca8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:35.537384 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0126a7a-8173-487b-b2a3-8545125ca8b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:35.537384 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0126a7a-8173-487b-b2a3-8545125ca8b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:14:35.537384 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0126a7a-8173-487b-b2a3-8545125ca8b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:35.562337 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0126a7a-8173-487b-b2a3-8545125ca8b6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:35.599301 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 868/1719] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:14:35.602847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2131e-7aa3-4437-b74a-38259c9f2ba1 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:35.618170 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2131e-7aa3-4437-b74a-38259c9f2ba1 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:35.640323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca5177-5b3a-4382-bca1-1eb1a030f418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:35.654255 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca5177-5b3a-4382-bca1-1eb1a030f418 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:35.654255 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca5177-5b3a-4382-bca1-1eb1a030f418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:35.654255 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca5177-5b3a-4382-bca1-1eb1a030f418 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:35.654255 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca5177-5b3a-4382-bca1-1eb1a030f418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:35.659818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca5177-5b3a-4382-bca1-1eb1a030f418 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:35.734504 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 852/1720] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:35 2022] POST /identity/v3/projects => generated 369 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:14:35.740112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0126a7a-8173-487b-b2a3-8545125ca8b6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:35.760592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0126a7a-8173-487b-b2a3-8545125ca8b6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:35.768979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4b33b-2430-4434-b843-3cb2994cfe1f None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:35.769638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4b33b-2430-4434-b843-3cb2994cfe1f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:35.782588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4b33b-2430-4434-b843-3cb2994cfe1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:35.840079 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:35.859948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e2a4b33b-2430-4434-b843-3cb2994cfe1f None None] MFA Rules not processed for user `95ac6a289fb345d3a98871f5e11f9414`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:35.864172 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e2a4b33b-2430-4434-b843-3cb2994cfe1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:36.057853 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203ec33-9ca6-4a0f-8d76-97ff5afd1e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:36.059145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203ec33-9ca6-4a0f-8d76-97ff5afd1e4f admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:36.059432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203ec33-9ca6-4a0f-8d76-97ff5afd1e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:36.059716 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203ec33-9ca6-4a0f-8d76-97ff5afd1e4f admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:36.060407 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203ec33-9ca6-4a0f-8d76-97ff5afd1e4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:36.071280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203ec33-9ca6-4a0f-8d76-97ff5afd1e4f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:36.126758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 853/1721] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:14:36.157058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 869/1722] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:35 2022] POST /identity/v3/users => generated 285 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:14:36.164945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4b33b-2430-4434-b843-3cb2994cfe1f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:36.182359 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203ec33-9ca6-4a0f-8d76-97ff5afd1e4f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:36.186203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4203ec33-9ca6-4a0f-8d76-97ff5afd1e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:36.207623 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4b33b-2430-4434-b843-3cb2994cfe1f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:36.214115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836330ff-2d15-4b16-ab6f-b3a63163c82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:36.215001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836330ff-2d15-4b16-ab6f-b3a63163c82d admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:36.215269 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836330ff-2d15-4b16-ab6f-b3a63163c82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:36.215554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836330ff-2d15-4b16-ab6f-b3a63163c82d admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:36.216252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836330ff-2d15-4b16-ab6f-b3a63163c82d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:36.217952 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836330ff-2d15-4b16-ab6f-b3a63163c82d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:36.234723 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 854/1723] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:36.236176 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203ec33-9ca6-4a0f-8d76-97ff5afd1e4f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:36.240327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836330ff-2d15-4b16-ab6f-b3a63163c82d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:36.257568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176667f3-a4d2-4206-a144-c9e0016f2bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:36.258292 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176667f3-a4d2-4206-a144-c9e0016f2bbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:36.258579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176667f3-a4d2-4206-a144-c9e0016f2bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:36.258885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176667f3-a4d2-4206-a144-c9e0016f2bbb admin admin] PATH_INFO: `/v3/projects/7bc923f0767745309a0b4471ff64bae0` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:36.260571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176667f3-a4d2-4206-a144-c9e0016f2bbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bc923f0767745309a0b4471ff64bae0)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:36.262911 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176667f3-a4d2-4206-a144-c9e0016f2bbb admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:36.264114 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836330ff-2d15-4b16-ab6f-b3a63163c82d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:36.311891 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6654cb-75ce-49ac-b438-b465ba86d331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:36.314505 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6654cb-75ce-49ac-b438-b465ba86d331 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:36.315686 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6654cb-75ce-49ac-b438-b465ba86d331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:36.316153 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6654cb-75ce-49ac-b438-b465ba86d331 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:36.317289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6654cb-75ce-49ac-b438-b465ba86d331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:36.322603 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6654cb-75ce-49ac-b438-b465ba86d331 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:36.359327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 855/1724] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:14:36.363862 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6654cb-75ce-49ac-b438-b465ba86d331 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:36.374487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-176667f3-a4d2-4206-a144-c9e0016f2bbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bc923f0767745309a0b4471ff64bae0', 'request_id': 'req-176667f3-a4d2-4206-a144-c9e0016f2bbb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:36.396995 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-176667f3-a4d2-4206-a144-c9e0016f2bbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bc923f0767745309a0b4471ff64bae0', 'request_id': 'req-176667f3-a4d2-4206-a144-c9e0016f2bbb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:36.406522 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6654cb-75ce-49ac-b438-b465ba86d331 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:36.408465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-176667f3-a4d2-4206-a144-c9e0016f2bbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bc923f0767745309a0b4471ff64bae0', 'request_id': 'req-176667f3-a4d2-4206-a144-c9e0016f2bbb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:36.427383 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-176667f3-a4d2-4206-a144-c9e0016f2bbb admin admin] The token cache is being invalidate because project 7bc923f0767745309a0b4471ff64bae0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:14:36.427800 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-176667f3-a4d2-4206-a144-c9e0016f2bbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:36.428878 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-176667f3-a4d2-4206-a144-c9e0016f2bbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:36.435556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 870/1725] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:14:36 2022] DELETE /identity/v3/projects/7bc923f0767745309a0b4471ff64bae0 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:14:36.437990 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f468e-c66e-49ae-a1ac-e31818c60cfa None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:36.437990 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f468e-c66e-49ae-a1ac-e31818c60cfa None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:36.438372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f468e-c66e-49ae-a1ac-e31818c60cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:36.466843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-5e9f468e-c66e-49ae-a1ac-e31818c60cfa None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:36.468629 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5e9f468e-c66e-49ae-a1ac-e31818c60cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:36.654959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c30603-68f8-424b-b3a7-07aee38c002e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:36.656836 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c30603-68f8-424b-b3a7-07aee38c002e admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:36.657513 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c30603-68f8-424b-b3a7-07aee38c002e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:36.658262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c30603-68f8-424b-b3a7-07aee38c002e admin admin] PATH_INFO: `/v3/projects/0665c54871b848f78e47b948dc00d53c/users/72495fdf53b9430abfedef2a8c068455/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:36.705907 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 871/1726] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:36.709495 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdd836-6750-497f-a539-cae0f0d7e237 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:36.709982 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdd836-6750-497f-a539-cae0f0d7e237 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:36.711521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdd836-6750-497f-a539-cae0f0d7e237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:36.757405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c30603-68f8-424b-b3a7-07aee38c002e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0665c54871b848f78e47b948dc00d53c, user_id=72495fdf53b9430abfedef2a8c068455, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:36.779500 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c30603-68f8-424b-b3a7-07aee38c002e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:36.808679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-49cdd836-6750-497f-a539-cae0f0d7e237 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:36.814401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-49cdd836-6750-497f-a539-cae0f0d7e237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:36.830306 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:36.873078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 856/1727] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:36 2022] PUT /identity/v3/projects/0665c54871b848f78e47b948dc00d53c/users/72495fdf53b9430abfedef2a8c068455/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:14:36.900670 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 872/1728] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:36.904799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdd836-6750-497f-a539-cae0f0d7e237 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:36.906236 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c85d1-3a12-42d6-a77c-caa3248709ac None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:36.907086 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c85d1-3a12-42d6-a77c-caa3248709ac None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:36.907879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c85d1-3a12-42d6-a77c-caa3248709ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:36.947400 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdd836-6750-497f-a539-cae0f0d7e237 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:37.042250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-913c85d1-3a12-42d6-a77c-caa3248709ac None None] MFA Rules not processed for user `72495fdf53b9430abfedef2a8c068455`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:37.044655 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:37.048157 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-913c85d1-3a12-42d6-a77c-caa3248709ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:37.177222 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328282d-5363-402c-a0bf-73d165e68814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:37.178556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328282d-5363-402c-a0bf-73d165e68814 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:37.179022 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328282d-5363-402c-a0bf-73d165e68814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:37.179442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328282d-5363-402c-a0bf-73d165e68814 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:37.181036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0328282d-5363-402c-a0bf-73d165e68814 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:14:37.181794 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0328282d-5363-402c-a0bf-73d165e68814 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:37.183999 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0328282d-5363-402c-a0bf-73d165e68814 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:37.223866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 873/1729] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:14:37.230117 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328282d-5363-402c-a0bf-73d165e68814 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:37.278854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328282d-5363-402c-a0bf-73d165e68814 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:37.300657 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fed9f5-df90-4871-9986-2d68f885df7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:37.301673 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fed9f5-df90-4871-9986-2d68f885df7c admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:37.302157 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fed9f5-df90-4871-9986-2d68f885df7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:37.303220 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fed9f5-df90-4871-9986-2d68f885df7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:37.304062 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fed9f5-df90-4871-9986-2d68f885df7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:37.327145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fed9f5-df90-4871-9986-2d68f885df7c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:37.456377 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 857/1730] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 561 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:14:37.462225 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c85d1-3a12-42d6-a77c-caa3248709ac None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:37.463842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-913c85d1-3a12-42d6-a77c-caa3248709ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:37.503049 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c85d1-3a12-42d6-a77c-caa3248709ac None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:37.522897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7eaf15-5ba4-4d4f-92e7-904e1d4ddc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:37.523813 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7eaf15-5ba4-4d4f-92e7-904e1d4ddc8c admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:37.524307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7eaf15-5ba4-4d4f-92e7-904e1d4ddc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:37.524752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7eaf15-5ba4-4d4f-92e7-904e1d4ddc8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:37.525684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7eaf15-5ba4-4d4f-92e7-904e1d4ddc8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:37.528040 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7eaf15-5ba4-4d4f-92e7-904e1d4ddc8c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:37.550750 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 874/1731] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:37 2022] POST /identity/v3/projects => generated 391 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:37.552194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fed9f5-df90-4871-9986-2d68f885df7c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:37.618337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 858/1732] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:37 2022] POST /identity/v3/projects => generated 369 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:14:37.625083 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7eaf15-5ba4-4d4f-92e7-904e1d4ddc8c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:37.860204 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fed9f5-df90-4871-9986-2d68f885df7c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:37.926246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7eaf15-5ba4-4d4f-92e7-904e1d4ddc8c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:37.944129 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9d213-7c94-416d-9362-b648e6237b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:37.945973 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9d213-7c94-416d-9362-b648e6237b8e admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:37.946873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9d213-7c94-416d-9362-b648e6237b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:37.947564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9d213-7c94-416d-9362-b648e6237b8e admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:37.951348 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9d213-7c94-416d-9362-b648e6237b8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:37.958770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9d213-7c94-416d-9362-b648e6237b8e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:38.068436 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 859/1733] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:37 2022] POST /identity/v3/users => generated 291 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:14:38.073850 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9d213-7c94-416d-9362-b648e6237b8e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:38.108945 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af9e2e-de49-46d8-af2e-f5887ae66a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:38.109809 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af9e2e-de49-46d8-af2e-f5887ae66a72 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:38.110565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af9e2e-de49-46d8-af2e-f5887ae66a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:38.113377 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9d213-7c94-416d-9362-b648e6237b8e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:38.113931 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af9e2e-de49-46d8-af2e-f5887ae66a72 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:38.115447 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af9e2e-de49-46d8-af2e-f5887ae66a72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:38.119576 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af9e2e-de49-46d8-af2e-f5887ae66a72 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:38.131533 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61df727-bd95-443a-b00c-e4e50fd11636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:38.133248 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61df727-bd95-443a-b00c-e4e50fd11636 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:38.134520 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61df727-bd95-443a-b00c-e4e50fd11636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:38.135859 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61df727-bd95-443a-b00c-e4e50fd11636 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:38.137306 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61df727-bd95-443a-b00c-e4e50fd11636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:38.141041 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61df727-bd95-443a-b00c-e4e50fd11636 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:38.219505 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 860/1734] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:38.223889 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61df727-bd95-443a-b00c-e4e50fd11636 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:38.240029 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 875/1735] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:37 2022] POST /identity/v3/users => generated 296 bytes in 687 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:38.242128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af9e2e-de49-46d8-af2e-f5887ae66a72 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:38.318787 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61df727-bd95-443a-b00c-e4e50fd11636 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:38.362478 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af9e2e-de49-46d8-af2e-f5887ae66a72 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:38.364922 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e808b-952c-4295-a06d-4fc396ce1006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:38.367268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e808b-952c-4295-a06d-4fc396ce1006 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:38.368419 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e808b-952c-4295-a06d-4fc396ce1006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:38.368798 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e808b-952c-4295-a06d-4fc396ce1006 admin admin] PATH_INFO: `/v3/projects/b73585d5d3f4491a8197be34e2b758c1/users/9d8c18dae2b14d6c93ddb0ba4d5480d2/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:38.412240 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c008b5a-ab73-4371-bde6-ac7f35d2a8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:38.417330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c008b5a-ab73-4371-bde6-ac7f35d2a8fb admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:38.417330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c008b5a-ab73-4371-bde6-ac7f35d2a8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:38.417330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c008b5a-ab73-4371-bde6-ac7f35d2a8fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:38.417330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c008b5a-ab73-4371-bde6-ac7f35d2a8fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:38.436442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c008b5a-ab73-4371-bde6-ac7f35d2a8fb admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:38.500006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e808b-952c-4295-a06d-4fc396ce1006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b73585d5d3f4491a8197be34e2b758c1, user_id=9d8c18dae2b14d6c93ddb0ba4d5480d2, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:38.506920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e808b-952c-4295-a06d-4fc396ce1006 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:38.527306 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 876/1736] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:38.532474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c008b5a-ab73-4371-bde6-ac7f35d2a8fb admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:38.543652 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 861/1737] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:38 2022] PUT /identity/v3/projects/b73585d5d3f4491a8197be34e2b758c1/users/9d8c18dae2b14d6c93ddb0ba4d5480d2/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:38.544732 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e808b-952c-4295-a06d-4fc396ce1006 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:38.610472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c008b5a-ab73-4371-bde6-ac7f35d2a8fb admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:38.647123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:38.692860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:38.823936 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e808b-952c-4295-a06d-4fc396ce1006 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:38.827926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d93c10-663d-426b-b450-1d84e4c56cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:38.828571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d93c10-663d-426b-b450-1d84e4c56cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:38.828945 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d93c10-663d-426b-b450-1d84e4c56cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:38.829338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d93c10-663d-426b-b450-1d84e4c56cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:38.829872 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d93c10-663d-426b-b450-1d84e4c56cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:38.833901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d93c10-663d-426b-b450-1d84e4c56cd4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:38.860401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 877/1738] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:14:38.871138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d93c10-663d-426b-b450-1d84e4c56cd4 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:38.875352 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ded09-dd70-4790-859e-2c6d7a859eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:38.877845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ded09-dd70-4790-859e-2c6d7a859eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:38.880691 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ded09-dd70-4790-859e-2c6d7a859eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:38.882321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ded09-dd70-4790-859e-2c6d7a859eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:38.884787 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ded09-dd70-4790-859e-2c6d7a859eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:38.889784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d93c10-663d-426b-b450-1d84e4c56cd4 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:38.890018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ded09-dd70-4790-859e-2c6d7a859eb5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:38.910761 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf54967-b437-4311-bbec-0aa167663183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:38.911479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf54967-b437-4311-bbec-0aa167663183 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:38.911762 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf54967-b437-4311-bbec-0aa167663183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:38.912046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf54967-b437-4311-bbec-0aa167663183 admin admin] PATH_INFO: `/v3/projects/d8cfeaedaf1b4a2ea345294f99260e55/users/87a9e1d33b2343d599123ce1a660dcae/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:38.954470 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf54967-b437-4311-bbec-0aa167663183 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8cfeaedaf1b4a2ea345294f99260e55, user_id=87a9e1d33b2343d599123ce1a660dcae, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:38.968827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf54967-b437-4311-bbec-0aa167663183 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:38.993164 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 862/1739] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 449 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:38.995365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ded09-dd70-4790-859e-2c6d7a859eb5 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:39.023891 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 878/1740] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:38 2022] PUT /identity/v3/projects/d8cfeaedaf1b4a2ea345294f99260e55/users/87a9e1d33b2343d599123ce1a660dcae/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:14:39.048868 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005237a-5fc1-4289-a5c5-7ed82d0b2f82 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:39.051816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005237a-5fc1-4289-a5c5-7ed82d0b2f82 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:39.053743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005237a-5fc1-4289-a5c5-7ed82d0b2f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:39.082761 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ded09-dd70-4790-859e-2c6d7a859eb5 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:39.303558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4005237a-5fc1-4289-a5c5-7ed82d0b2f82 None None] MFA Rules not processed for user `87a9e1d33b2343d599123ce1a660dcae`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:39.304757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4005237a-5fc1-4289-a5c5-7ed82d0b2f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:39.526299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7d4b6-8d0f-4b10-b3ef-fe9e6227efba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:39.532270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7d4b6-8d0f-4b10-b3ef-fe9e6227efba admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:39.535812 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7d4b6-8d0f-4b10-b3ef-fe9e6227efba admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:39.538581 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7d4b6-8d0f-4b10-b3ef-fe9e6227efba admin admin] PATH_INFO: `/v3/projects/b73585d5d3f4491a8197be34e2b758c1/users/9d8c18dae2b14d6c93ddb0ba4d5480d2/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:39.558412 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7d4b6-8d0f-4b10-b3ef-fe9e6227efba admin admin] RBAC: Authorizing `identity:create_grant(project_id=b73585d5d3f4491a8197be34e2b758c1, user_id=9d8c18dae2b14d6c93ddb0ba4d5480d2, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:39.587430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7d4b6-8d0f-4b10-b3ef-fe9e6227efba admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:39.631560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 879/1741] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 585 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:39.633290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005237a-5fc1-4289-a5c5-7ed82d0b2f82 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:39.636589 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4005237a-5fc1-4289-a5c5-7ed82d0b2f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:39.676057 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005237a-5fc1-4289-a5c5-7ed82d0b2f82 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:39.732617 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 863/1742] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:38 2022] PUT /identity/v3/projects/b73585d5d3f4491a8197be34e2b758c1/users/9d8c18dae2b14d6c93ddb0ba4d5480d2/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:39.742369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7d4b6-8d0f-4b10-b3ef-fe9e6227efba admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:39.914775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5410b6-9090-4e3c-a055-cdf0c6f5d159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:39.915668 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5410b6-9090-4e3c-a055-cdf0c6f5d159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:39.915873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5410b6-9090-4e3c-a055-cdf0c6f5d159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:39.916152 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5410b6-9090-4e3c-a055-cdf0c6f5d159 admin admin] PATH_INFO: `/v3/users/12d8f855c8fc4dab9d97afc6f7bab933` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:39.930685 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5410b6-9090-4e3c-a055-cdf0c6f5d159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12d8f855c8fc4dab9d97afc6f7bab933)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:39.934940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5410b6-9090-4e3c-a055-cdf0c6f5d159 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:40.100216 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:40.104462 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7d4b6-8d0f-4b10-b3ef-fe9e6227efba admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:40.155384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70542f33-bbca-4609-9360-ab34103003e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:40.161101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70542f33-bbca-4609-9360-ab34103003e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:40.161663 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70542f33-bbca-4609-9360-ab34103003e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:40.163168 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70542f33-bbca-4609-9360-ab34103003e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:40.163372 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70542f33-bbca-4609-9360-ab34103003e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:40.178714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-aa5410b6-9090-4e3c-a055-cdf0c6f5d159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12d8f855c8fc4dab9d97afc6f7bab933'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:40.202806 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70542f33-bbca-4609-9360-ab34103003e3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:40.209459 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-aa5410b6-9090-4e3c-a055-cdf0c6f5d159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12d8f855c8fc4dab9d97afc6f7bab933'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:40.266465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-aa5410b6-9090-4e3c-a055-cdf0c6f5d159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12d8f855c8fc4dab9d97afc6f7bab933'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:40.269593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-aa5410b6-9090-4e3c-a055-cdf0c6f5d159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12d8f855c8fc4dab9d97afc6f7bab933'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:40.288008 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-aa5410b6-9090-4e3c-a055-cdf0c6f5d159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12d8f855c8fc4dab9d97afc6f7bab933'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:40.331647 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 880/1743] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:14:39 2022] DELETE /identity/v3/users/12d8f855c8fc4dab9d97afc6f7bab933 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:40.343667 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60008e-cec7-4438-9f2a-f66f2078674f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:40.344562 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60008e-cec7-4438-9f2a-f66f2078674f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:40.349846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60008e-cec7-4438-9f2a-f66f2078674f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:40.411448 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 864/1744] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:39 2022] POST /identity/v3/projects => generated 393 bytes in 678 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:40.413428 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea723b-804b-4fa2-8451-657fa3b57341 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:40.413897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea723b-804b-4fa2-8451-657fa3b57341 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:40.414344 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea723b-804b-4fa2-8451-657fa3b57341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:40.499575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-1b60008e-cec7-4438-9f2a-f66f2078674f None None] MFA Rules not processed for user `9d8c18dae2b14d6c93ddb0ba4d5480d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:40.509318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1b60008e-cec7-4438-9f2a-f66f2078674f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:40.513193 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-41ea723b-804b-4fa2-8451-657fa3b57341 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:40.516204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-41ea723b-804b-4fa2-8451-657fa3b57341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:40.634488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:40.781040 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 865/1745] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:40.786642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea723b-804b-4fa2-8451-657fa3b57341 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:40.797210 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-41ea723b-804b-4fa2-8451-657fa3b57341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:40.881974 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea723b-804b-4fa2-8451-657fa3b57341 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:40.926950 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 881/1746] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 594 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:40.928708 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60008e-cec7-4438-9f2a-f66f2078674f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:40.936887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1b60008e-cec7-4438-9f2a-f66f2078674f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:40.938409 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541aeae-c796-414b-988c-5e19cc07f8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:40.942255 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541aeae-c796-414b-988c-5e19cc07f8de admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:40.943738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541aeae-c796-414b-988c-5e19cc07f8de admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:40.945113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541aeae-c796-414b-988c-5e19cc07f8de admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:40.948200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9541aeae-c796-414b-988c-5e19cc07f8de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:40.953546 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9541aeae-c796-414b-988c-5e19cc07f8de admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:41.153130 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 866/1747] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:40 2022] POST /identity/v3/users => generated 303 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:41.155695 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541aeae-c796-414b-988c-5e19cc07f8de admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:41.160791 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9541aeae-c796-414b-988c-5e19cc07f8de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:41.378625 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60008e-cec7-4438-9f2a-f66f2078674f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:41.384100 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541aeae-c796-414b-988c-5e19cc07f8de admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:41.389884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac9271-9413-4580-90d5-56f5748f819c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:41.391339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac9271-9413-4580-90d5-56f5748f819c service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:41.395566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac9271-9413-4580-90d5-56f5748f819c service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:41.397376 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac9271-9413-4580-90d5-56f5748f819c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:41.401158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ba40c-0618-4791-a939-f9bf452339a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:41.403827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ba40c-0618-4791-a939-f9bf452339a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:41.408765 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ba40c-0618-4791-a939-f9bf452339a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:41.411911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ba40c-0618-4791-a939-f9bf452339a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:41.433994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac9271-9413-4580-90d5-56f5748f819c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:41.437139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac9271-9413-4580-90d5-56f5748f819c service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:41.447299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ba40c-0618-4791-a939-f9bf452339a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:41.456428 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ba40c-0618-4791-a939-f9bf452339a6 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:41.472504 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 882/1748] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:41.475498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac9271-9413-4580-90d5-56f5748f819c service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:41.482420 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 867/1749] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:41.501850 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d31050-39a8-4da2-b37d-084a64fb1814 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:41.505430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d31050-39a8-4da2-b37d-084a64fb1814 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:41.507251 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d31050-39a8-4da2-b37d-084a64fb1814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:41.521486 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac9271-9413-4580-90d5-56f5748f819c service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:41.566262 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219969a8-0d60-4821-abbd-ae554d08b816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:41.573802 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219969a8-0d60-4821-abbd-ae554d08b816 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:41.576295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219969a8-0d60-4821-abbd-ae554d08b816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:41.577999 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219969a8-0d60-4821-abbd-ae554d08b816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:41.584398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219969a8-0d60-4821-abbd-ae554d08b816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:41.607104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219969a8-0d60-4821-abbd-ae554d08b816 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:41.698877 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-97d31050-39a8-4da2-b37d-084a64fb1814 None None] MFA Rules not processed for user `989beaca146d4f9b950f333bd3567e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:41.700268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-97d31050-39a8-4da2-b37d-084a64fb1814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:41.718430 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 883/1750] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:41.720247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219969a8-0d60-4821-abbd-ae554d08b816 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:41.731663 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 868/1751] 10.222.0.20 () {58 vars in 1015 bytes} [Tue Sep 6 16:14:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:41.735108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d31050-39a8-4da2-b37d-084a64fb1814 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:41.767010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d31050-39a8-4da2-b37d-084a64fb1814 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:41.781902 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb8966-9cd9-42de-97b4-27bc7f0d98be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:41.783068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb8966-9cd9-42de-97b4-27bc7f0d98be service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:41.784121 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb8966-9cd9-42de-97b4-27bc7f0d98be service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:41.784560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb8966-9cd9-42de-97b4-27bc7f0d98be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:41.792482 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f2cb8966-9cd9-42de-97b4-27bc7f0d98be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:41.808234 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219969a8-0d60-4821-abbd-ae554d08b816 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:41.843988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2c335-46cd-4562-bf12-7934b15e6ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:41.848556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2c335-46cd-4562-bf12-7934b15e6ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:41.861946 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2c335-46cd-4562-bf12-7934b15e6ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:41.863153 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2c335-46cd-4562-bf12-7934b15e6ec4 admin admin] PATH_INFO: `/v3/projects/120861b4de3a430091fa1b8b0bf913d1/users/de4e5a300e0d409ba2267389f834bc48/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:41.963682 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2c335-46cd-4562-bf12-7934b15e6ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=120861b4de3a430091fa1b8b0bf913d1, user_id=de4e5a300e0d409ba2267389f834bc48, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:41.984206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2c335-46cd-4562-bf12-7934b15e6ec4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:42.008412 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:42.059278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 884/1752] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:41 2022] PUT /identity/v3/projects/120861b4de3a430091fa1b8b0bf913d1/users/de4e5a300e0d409ba2267389f834bc48/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:42.062780 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2c335-46cd-4562-bf12-7934b15e6ec4 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:42.071864 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1bc2c335-46cd-4562-bf12-7934b15e6ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:42.124639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb8966-9cd9-42de-97b4-27bc7f0d98be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:42.140586 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb8966-9cd9-42de-97b4-27bc7f0d98be service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:42.223283 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 869/1753] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:42.228618 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb8966-9cd9-42de-97b4-27bc7f0d98be service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:42.314727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb8966-9cd9-42de-97b4-27bc7f0d98be service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:42.447668 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2c335-46cd-4562-bf12-7934b15e6ec4 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:42.463389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717bb94-0985-47ca-8797-a5f7a3f6d35e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:42.468093 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717bb94-0985-47ca-8797-a5f7a3f6d35e service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:42.468532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717bb94-0985-47ca-8797-a5f7a3f6d35e service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:42.468833 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717bb94-0985-47ca-8797-a5f7a3f6d35e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:42.500458 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717bb94-0985-47ca-8797-a5f7a3f6d35e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:42.502940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717bb94-0985-47ca-8797-a5f7a3f6d35e service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:42.583869 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 885/1754] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:42.590371 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717bb94-0985-47ca-8797-a5f7a3f6d35e service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:42.630278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717bb94-0985-47ca-8797-a5f7a3f6d35e service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:42.649331 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d5c17-64a9-4b18-97b2-089246192b38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:42.657975 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d5c17-64a9-4b18-97b2-089246192b38 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:42.660480 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d5c17-64a9-4b18-97b2-089246192b38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:42.661045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d5c17-64a9-4b18-97b2-089246192b38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:42.695600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d5c17-64a9-4b18-97b2-089246192b38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:42.698487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d5c17-64a9-4b18-97b2-089246192b38 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:42.711699 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:42.743109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 886/1755] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:42.776028 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82b47e-23c8-493a-9b80-9730f82afc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:42.788022 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82b47e-23c8-493a-9b80-9730f82afc38 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:42.791074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82b47e-23c8-493a-9b80-9730f82afc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:42.792520 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82b47e-23c8-493a-9b80-9730f82afc38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:42.795851 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82b47e-23c8-493a-9b80-9730f82afc38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:42.812975 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82b47e-23c8-493a-9b80-9730f82afc38 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:42.947467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 870/1756] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 720 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:42.952421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d5c17-64a9-4b18-97b2-089246192b38 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:42.981872 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d5c17-64a9-4b18-97b2-089246192b38 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:43.005044 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4070c17-edcc-490b-8077-bf6a372c9a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:43.008794 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4070c17-edcc-490b-8077-bf6a372c9a42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:43.009170 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4070c17-edcc-490b-8077-bf6a372c9a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:43.009461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4070c17-edcc-490b-8077-bf6a372c9a42 admin admin] PATH_INFO: `/v3/projects/120861b4de3a430091fa1b8b0bf913d1/users/de4e5a300e0d409ba2267389f834bc48/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:43.024973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4070c17-edcc-490b-8077-bf6a372c9a42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=120861b4de3a430091fa1b8b0bf913d1, user_id=de4e5a300e0d409ba2267389f834bc48, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:43.039904 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4070c17-edcc-490b-8077-bf6a372c9a42 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:43.109555 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 887/1757] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:42 2022] PUT /identity/v3/projects/120861b4de3a430091fa1b8b0bf913d1/users/de4e5a300e0d409ba2267389f834bc48/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:43.118452 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11484710-8375-4874-86cc-99a8d1589d30 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:43.119669 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11484710-8375-4874-86cc-99a8d1589d30 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:43.120468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11484710-8375-4874-86cc-99a8d1589d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:43.260522 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-11484710-8375-4874-86cc-99a8d1589d30 None None] MFA Rules not processed for user `de4e5a300e0d409ba2267389f834bc48`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:43.270696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-11484710-8375-4874-86cc-99a8d1589d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:43.535258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 871/1758] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:43 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:43.822142 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4070c17-edcc-490b-8077-bf6a372c9a42 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:44.345203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4070c17-edcc-490b-8077-bf6a372c9a42 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:44.366810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477909f0-2800-4757-8805-12498e17f027 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:44.368693 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477909f0-2800-4757-8805-12498e17f027 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:44.370303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477909f0-2800-4757-8805-12498e17f027 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:44.370810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477909f0-2800-4757-8805-12498e17f027 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:44.413055 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477909f0-2800-4757-8805-12498e17f027 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:44.416677 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477909f0-2800-4757-8805-12498e17f027 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:44.429972 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11484710-8375-4874-86cc-99a8d1589d30 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:44.432954 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-11484710-8375-4874-86cc-99a8d1589d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:44.467168 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 888/1759] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:43 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:14:44.978712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11484710-8375-4874-86cc-99a8d1589d30 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:44.989777 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4416f4-8258-4931-8538-d0dd24137f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:44.991749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4416f4-8258-4931-8538-d0dd24137f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:44.993356 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4416f4-8258-4931-8538-d0dd24137f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:44.993893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4416f4-8258-4931-8538-d0dd24137f43 admin admin] PATH_INFO: `/v3/projects/8eb61062f4ab48a49ee3b5f8d02ce761` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:44.999890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4416f4-8258-4931-8538-d0dd24137f43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eb61062f4ab48a49ee3b5f8d02ce761)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:45.006976 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4416f4-8258-4931-8538-d0dd24137f43 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:45.096544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477909f0-2800-4757-8805-12498e17f027 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:45.110705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477909f0-2800-4757-8805-12498e17f027 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:45.126917 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1229010-34c9-49d2-8de4-a7991bb3be1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:45.128267 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1229010-34c9-49d2-8de4-a7991bb3be1e service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:45.128754 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1229010-34c9-49d2-8de4-a7991bb3be1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:45.129187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1229010-34c9-49d2-8de4-a7991bb3be1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:45.132705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a1229010-34c9-49d2-8de4-a7991bb3be1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:45.197443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4d4416f4-8258-4931-8538-d0dd24137f43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eb61062f4ab48a49ee3b5f8d02ce761', 'request_id': 'req-4d4416f4-8258-4931-8538-d0dd24137f43'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:45.238790 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4d4416f4-8258-4931-8538-d0dd24137f43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eb61062f4ab48a49ee3b5f8d02ce761', 'request_id': 'req-4d4416f4-8258-4931-8538-d0dd24137f43'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:45.249956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4d4416f4-8258-4931-8538-d0dd24137f43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eb61062f4ab48a49ee3b5f8d02ce761', 'request_id': 'req-4d4416f4-8258-4931-8538-d0dd24137f43'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:45.286376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4d4416f4-8258-4931-8538-d0dd24137f43 admin admin] The token cache is being invalidate because project 8eb61062f4ab48a49ee3b5f8d02ce761 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:14:45.293517 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4d4416f4-8258-4931-8538-d0dd24137f43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:45.294611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4d4416f4-8258-4931-8538-d0dd24137f43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:45.301463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 872/1760] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:14:44 2022] DELETE /identity/v3/projects/8eb61062f4ab48a49ee3b5f8d02ce761 => generated 0 bytes in 878 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:45.310539 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fee52-f2d7-401f-a312-2fb065040379 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:45.316162 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fee52-f2d7-401f-a312-2fb065040379 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:45.321748 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fee52-f2d7-401f-a312-2fb065040379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:45.380817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:45.439132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d27fee52-f2d7-401f-a312-2fb065040379 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:45.452993 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d27fee52-f2d7-401f-a312-2fb065040379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:45.576339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1229010-34c9-49d2-8de4-a7991bb3be1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:45.592832 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1229010-34c9-49d2-8de4-a7991bb3be1e service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:45.660602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 889/1761] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:45 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:45.662494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1229010-34c9-49d2-8de4-a7991bb3be1e service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:45.672742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a1229010-34c9-49d2-8de4-a7991bb3be1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:45.764158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1229010-34c9-49d2-8de4-a7991bb3be1e service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:45.809044 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:45.896456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 873/1762] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 578 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:45.899173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f621dc-b28b-4616-b143-a83c92d14eeb None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:45.899650 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f621dc-b28b-4616-b143-a83c92d14eeb None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:45.900202 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f621dc-b28b-4616-b143-a83c92d14eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:46.040709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d1f621dc-b28b-4616-b143-a83c92d14eeb None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:46.057212 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d1f621dc-b28b-4616-b143-a83c92d14eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:46.112256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 874/1763] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:14:46.113902 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f621dc-b28b-4616-b143-a83c92d14eeb None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:46.153932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e16ae-0ec3-443f-9fce-72ad24ba1384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:46.154318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f621dc-b28b-4616-b143-a83c92d14eeb None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:46.156426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e16ae-0ec3-443f-9fce-72ad24ba1384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:46.157009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e16ae-0ec3-443f-9fce-72ad24ba1384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:46.157488 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e16ae-0ec3-443f-9fce-72ad24ba1384 admin admin] PATH_INFO: `/v3/users/e3c7750f19234336bf008c3474dffbf1` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:46.165736 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e16ae-0ec3-443f-9fce-72ad24ba1384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3c7750f19234336bf008c3474dffbf1)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:46.176203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e16ae-0ec3-443f-9fce-72ad24ba1384 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:46.181320 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606983c-1d3b-4dea-9b18-a878e7edcf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:46.183709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606983c-1d3b-4dea-9b18-a878e7edcf6b admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:46.184127 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606983c-1d3b-4dea-9b18-a878e7edcf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:46.184527 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606983c-1d3b-4dea-9b18-a878e7edcf6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:46.190375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606983c-1d3b-4dea-9b18-a878e7edcf6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:14:46.190375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606983c-1d3b-4dea-9b18-a878e7edcf6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:46.193904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606983c-1d3b-4dea-9b18-a878e7edcf6b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:46.256583 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 875/1764] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:14:46.261658 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606983c-1d3b-4dea-9b18-a878e7edcf6b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:46.323349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606983c-1d3b-4dea-9b18-a878e7edcf6b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:46.374419 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c386d37-0b9e-4d49-af2b-f2593ef2cbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:46.381246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c386d37-0b9e-4d49-af2b-f2593ef2cbea admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:46.381802 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c386d37-0b9e-4d49-af2b-f2593ef2cbea admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:46.382281 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c386d37-0b9e-4d49-af2b-f2593ef2cbea admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:46.387342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c386d37-0b9e-4d49-af2b-f2593ef2cbea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:46.399866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-399e16ae-0ec3-443f-9fce-72ad24ba1384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3c7750f19234336bf008c3474dffbf1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:46.400174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c386d37-0b9e-4d49-af2b-f2593ef2cbea admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:46.447988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-399e16ae-0ec3-443f-9fce-72ad24ba1384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3c7750f19234336bf008c3474dffbf1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:46.518552 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-399e16ae-0ec3-443f-9fce-72ad24ba1384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3c7750f19234336bf008c3474dffbf1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:46.520593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-399e16ae-0ec3-443f-9fce-72ad24ba1384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3c7750f19234336bf008c3474dffbf1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:46.571757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-399e16ae-0ec3-443f-9fce-72ad24ba1384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3c7750f19234336bf008c3474dffbf1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:46.582905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 876/1765] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:46 2022] POST /identity/v3/projects => generated 403 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:46.584711 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c386d37-0b9e-4d49-af2b-f2593ef2cbea admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:46.593535 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4c386d37-0b9e-4d49-af2b-f2593ef2cbea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:46.609422 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 890/1766] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:14:45 2022] DELETE /identity/v3/users/e3c7750f19234336bf008c3474dffbf1 => generated 0 bytes in 947 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:46.616096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8968e6-358a-48ec-bd56-cbfbd777b63e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:46.617969 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8968e6-358a-48ec-bd56-cbfbd777b63e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:46.618511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8968e6-358a-48ec-bd56-cbfbd777b63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:46.772863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4b8968e6-358a-48ec-bd56-cbfbd777b63e None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:46.777933 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4b8968e6-358a-48ec-bd56-cbfbd777b63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:46.982100 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 891/1767] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:47.010197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c386d37-0b9e-4d49-af2b-f2593ef2cbea admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:47.027308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406e80f-b46a-4ccd-a0d1-3627f137b904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:47.027951 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406e80f-b46a-4ccd-a0d1-3627f137b904 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:47.028228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406e80f-b46a-4ccd-a0d1-3627f137b904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:47.028802 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406e80f-b46a-4ccd-a0d1-3627f137b904 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:47.029980 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406e80f-b46a-4ccd-a0d1-3627f137b904 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:47.034025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406e80f-b46a-4ccd-a0d1-3627f137b904 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:47.115467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 877/1768] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:46 2022] POST /identity/v3/users => generated 302 bytes in 531 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:47.117727 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8968e6-358a-48ec-bd56-cbfbd777b63e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:47.130354 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406e80f-b46a-4ccd-a0d1-3627f137b904 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:47.133651 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0406e80f-b46a-4ccd-a0d1-3627f137b904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:47.176974 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8968e6-358a-48ec-bd56-cbfbd777b63e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:47.264925 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f08c5e-73d7-4f65-a809-bbb8d03fe445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:47.266079 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f08c5e-73d7-4f65-a809-bbb8d03fe445 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:47.266428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f08c5e-73d7-4f65-a809-bbb8d03fe445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:47.296516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f08c5e-73d7-4f65-a809-bbb8d03fe445 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:47.309258 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f08c5e-73d7-4f65-a809-bbb8d03fe445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:47.328333 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f08c5e-73d7-4f65-a809-bbb8d03fe445 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:47.371500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 892/1769] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:14:47.378797 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f08c5e-73d7-4f65-a809-bbb8d03fe445 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:47.419325 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f08c5e-73d7-4f65-a809-bbb8d03fe445 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:47.435360 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4b95a-07c5-453b-87e1-0f5e729e28b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:47.436487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4b95a-07c5-453b-87e1-0f5e729e28b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:47.438674 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4b95a-07c5-453b-87e1-0f5e729e28b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:47.438974 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4b95a-07c5-453b-87e1-0f5e729e28b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:47.441108 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4b95a-07c5-453b-87e1-0f5e729e28b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:47.446736 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4b95a-07c5-453b-87e1-0f5e729e28b7 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:47.488078 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 893/1770] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:47.510579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4b95a-07c5-453b-87e1-0f5e729e28b7 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:47.603244 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4b95a-07c5-453b-87e1-0f5e729e28b7 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:47.632650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f06d21-3392-4dc6-90ec-8578924a298f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:47.633759 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f06d21-3392-4dc6-90ec-8578924a298f admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:47.634321 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f06d21-3392-4dc6-90ec-8578924a298f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:47.645101 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f06d21-3392-4dc6-90ec-8578924a298f admin admin] PATH_INFO: `/v3/projects/8368e465a9a148adaf12244d9b96a178/users/550277b863fd4797a741b77f716bef56/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:47.653381 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406e80f-b46a-4ccd-a0d1-3627f137b904 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:47.675039 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0d8d5-3621-4a45-9215-5872182ca548 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:47.686552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0d8d5-3621-4a45-9215-5872182ca548 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:47.690425 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0d8d5-3621-4a45-9215-5872182ca548 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:47.691421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0d8d5-3621-4a45-9215-5872182ca548 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:47.691635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:47.742652 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f06d21-3392-4dc6-90ec-8578924a298f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8368e465a9a148adaf12244d9b96a178, user_id=550277b863fd4797a741b77f716bef56, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:47.748794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0d8d5-3621-4a45-9215-5872182ca548 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:47.761572 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f06d21-3392-4dc6-90ec-8578924a298f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:47.769164 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0d8d5-3621-4a45-9215-5872182ca548 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:47.788435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:47.835870 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 878/1771] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 706 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:47.845469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 894/1772] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:47 2022] PUT /identity/v3/projects/8368e465a9a148adaf12244d9b96a178/users/550277b863fd4797a741b77f716bef56/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:47.853759 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a8a71-563b-4686-a755-2addd3836e75 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:47.855110 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a8a71-563b-4686-a755-2addd3836e75 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:47.856510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a8a71-563b-4686-a755-2addd3836e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:47.980178 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-6c1a8a71-563b-4686-a755-2addd3836e75 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:47.987696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6c1a8a71-563b-4686-a755-2addd3836e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:48.355412 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 879/1773] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 504 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:48.477769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f06d21-3392-4dc6-90ec-8578924a298f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:48.481599 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e0f06d21-3392-4dc6-90ec-8578924a298f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:48.697000 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f06d21-3392-4dc6-90ec-8578924a298f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:48.705882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951e563-1b66-4221-ab5b-313462529dd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:48.706821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951e563-1b66-4221-ab5b-313462529dd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:48.707271 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951e563-1b66-4221-ab5b-313462529dd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:48.707805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951e563-1b66-4221-ab5b-313462529dd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:48.723820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951e563-1b66-4221-ab5b-313462529dd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:48.726717 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951e563-1b66-4221-ab5b-313462529dd0 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:48.747557 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 895/1774] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:50.660601 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4eac62-031a-4f68-bcca-3d0f51938a5e None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:50.661719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4eac62-031a-4f68-bcca-3d0f51938a5e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:50.662247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4eac62-031a-4f68-bcca-3d0f51938a5e None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:50.666520 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 880/1775] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 6 16:14:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:14:50.787818 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722f1a1-7173-432d-8eeb-ab4f466df5fd None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:50.788647 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722f1a1-7173-432d-8eeb-ab4f466df5fd None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:50.788647 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722f1a1-7173-432d-8eeb-ab4f466df5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:50.814299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4eac62-031a-4f68-bcca-3d0f51938a5e None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:50.816517 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8a4eac62-031a-4f68-bcca-3d0f51938a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:50.852888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7722f1a1-7173-432d-8eeb-ab4f466df5fd None None] MFA Rules not processed for user `6afd4f6ffdae4b16aa5ba757e68c18c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:50.858324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7722f1a1-7173-432d-8eeb-ab4f466df5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:50.867972 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4eac62-031a-4f68-bcca-3d0f51938a5e None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:50.885812 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387c4c0-d6b4-4625-8752-566da90b0655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:50.887082 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387c4c0-d6b4-4625-8752-566da90b0655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:50.887594 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387c4c0-d6b4-4625-8752-566da90b0655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:50.891819 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387c4c0-d6b4-4625-8752-566da90b0655 admin admin] PATH_INFO: `/v3/projects/5e1b00dc91494f03853fb224ffeef11d` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:50.895595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387c4c0-d6b4-4625-8752-566da90b0655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e1b00dc91494f03853fb224ffeef11d)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:50.902653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387c4c0-d6b4-4625-8752-566da90b0655 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:51.114250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-6387c4c0-d6b4-4625-8752-566da90b0655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e1b00dc91494f03853fb224ffeef11d', 'request_id': 'req-6387c4c0-d6b4-4625-8752-566da90b0655'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:51.141366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-6387c4c0-d6b4-4625-8752-566da90b0655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e1b00dc91494f03853fb224ffeef11d', 'request_id': 'req-6387c4c0-d6b4-4625-8752-566da90b0655'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:51.167563 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-6387c4c0-d6b4-4625-8752-566da90b0655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e1b00dc91494f03853fb224ffeef11d', 'request_id': 'req-6387c4c0-d6b4-4625-8752-566da90b0655'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:51.214962 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-6387c4c0-d6b4-4625-8752-566da90b0655 admin admin] The token cache is being invalidate because project 5e1b00dc91494f03853fb224ffeef11d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:14:51.215280 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-6387c4c0-d6b4-4625-8752-566da90b0655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:51.220498 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-6387c4c0-d6b4-4625-8752-566da90b0655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:51.230051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 881/1776] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:14:50 2022] DELETE /identity/v3/projects/5e1b00dc91494f03853fb224ffeef11d => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:51.235570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387c4c0-d6b4-4625-8752-566da90b0655 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:51.239847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6387c4c0-d6b4-4625-8752-566da90b0655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:51.295881 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 896/1777] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 6 16:14:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 508 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:14:51.524214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387c4c0-d6b4-4625-8752-566da90b0655 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:51.541943 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cee799-8aa0-4246-9776-32725a35d864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:51.542620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cee799-8aa0-4246-9776-32725a35d864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:51.542922 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cee799-8aa0-4246-9776-32725a35d864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:51.543197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cee799-8aa0-4246-9776-32725a35d864 admin admin] PATH_INFO: `/v3/users/297ae31f4e324b6fb5940d799cfae5fc` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:51.544757 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cee799-8aa0-4246-9776-32725a35d864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=297ae31f4e324b6fb5940d799cfae5fc)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:51.548400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cee799-8aa0-4246-9776-32725a35d864 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:51.640866 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:51.711400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-09cee799-8aa0-4246-9776-32725a35d864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '297ae31f4e324b6fb5940d799cfae5fc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:51.732793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-09cee799-8aa0-4246-9776-32725a35d864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '297ae31f4e324b6fb5940d799cfae5fc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:51.770355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-09cee799-8aa0-4246-9776-32725a35d864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '297ae31f4e324b6fb5940d799cfae5fc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:51.773184 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-09cee799-8aa0-4246-9776-32725a35d864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '297ae31f4e324b6fb5940d799cfae5fc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:51.775086 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-09cee799-8aa0-4246-9776-32725a35d864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '297ae31f4e324b6fb5940d799cfae5fc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:51.820643 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 882/1778] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:14:51 2022] DELETE /identity/v3/users/297ae31f4e324b6fb5940d799cfae5fc => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:51.909957 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722f1a1-7173-432d-8eeb-ab4f466df5fd None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:51.913566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7722f1a1-7173-432d-8eeb-ab4f466df5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:51.977696 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722f1a1-7173-432d-8eeb-ab4f466df5fd None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:52.207930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b5d3f-036c-4a03-b995-be3d904ede11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:52.212032 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b5d3f-036c-4a03-b995-be3d904ede11 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:52.212952 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b5d3f-036c-4a03-b995-be3d904ede11 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:52.213150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b5d3f-036c-4a03-b995-be3d904ede11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:52.223210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bf6b5d3f-036c-4a03-b995-be3d904ede11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:52.277169 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b5d3f-036c-4a03-b995-be3d904ede11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:52.287651 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b5d3f-036c-4a03-b995-be3d904ede11 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:52.300234 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cee799-8aa0-4246-9776-32725a35d864 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:52.304540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-09cee799-8aa0-4246-9776-32725a35d864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:52.341782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 897/1779] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:52.542651 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cee799-8aa0-4246-9776-32725a35d864 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:52.550545 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04bb631-ffe2-4091-8773-1d6317980ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:52.551489 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04bb631-ffe2-4091-8773-1d6317980ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:52.551767 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04bb631-ffe2-4091-8773-1d6317980ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:52.552011 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04bb631-ffe2-4091-8773-1d6317980ffc admin admin] PATH_INFO: `/v3/users/72495fdf53b9430abfedef2a8c068455` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:52.553952 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04bb631-ffe2-4091-8773-1d6317980ffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=72495fdf53b9430abfedef2a8c068455)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:52.556202 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04bb631-ffe2-4091-8773-1d6317980ffc admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:52.744672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b04bb631-ffe2-4091-8773-1d6317980ffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72495fdf53b9430abfedef2a8c068455'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:52.779722 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b04bb631-ffe2-4091-8773-1d6317980ffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72495fdf53b9430abfedef2a8c068455'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:52.819619 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b04bb631-ffe2-4091-8773-1d6317980ffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72495fdf53b9430abfedef2a8c068455'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:52.821363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b04bb631-ffe2-4091-8773-1d6317980ffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72495fdf53b9430abfedef2a8c068455'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:52.822885 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b04bb631-ffe2-4091-8773-1d6317980ffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72495fdf53b9430abfedef2a8c068455'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:52.854053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 883/1780] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:14:52 2022] DELETE /identity/v3/users/72495fdf53b9430abfedef2a8c068455 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:52.866090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a96739-c31a-4273-93f1-954ca31b99ae None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:52.866581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a96739-c31a-4273-93f1-954ca31b99ae None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:52.873448 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a96739-c31a-4273-93f1-954ca31b99ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:52.906880 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:52.919820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d5a96739-c31a-4273-93f1-954ca31b99ae None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:52.921157 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d5a96739-c31a-4273-93f1-954ca31b99ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:53.132195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 898/1781] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:53.190175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04bb631-ffe2-4091-8773-1d6317980ffc admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:53.192095 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b04bb631-ffe2-4091-8773-1d6317980ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:53.278471 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:53.401577 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04bb631-ffe2-4091-8773-1d6317980ffc admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:53.417795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc81fd-45ff-4ed0-a18f-19f67f094f5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:53.417795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc81fd-45ff-4ed0-a18f-19f67f094f5b service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:53.417795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc81fd-45ff-4ed0-a18f-19f67f094f5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:53.417795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc81fd-45ff-4ed0-a18f-19f67f094f5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:53.452947 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc81fd-45ff-4ed0-a18f-19f67f094f5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:53.465038 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc81fd-45ff-4ed0-a18f-19f67f094f5b service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:53.503374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 884/1782] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:53.949402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a96739-c31a-4273-93f1-954ca31b99ae None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:53.952575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d5a96739-c31a-4273-93f1-954ca31b99ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:53.980189 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a96739-c31a-4273-93f1-954ca31b99ae None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:53.997370 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83030b5-b17f-43c3-92d0-e7f1469ae9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:53.999110 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83030b5-b17f-43c3-92d0-e7f1469ae9c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:54.000687 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83030b5-b17f-43c3-92d0-e7f1469ae9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:54.001943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83030b5-b17f-43c3-92d0-e7f1469ae9c6 admin admin] PATH_INFO: `/v3/users/87a9e1d33b2343d599123ce1a660dcae` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:54.005102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83030b5-b17f-43c3-92d0-e7f1469ae9c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87a9e1d33b2343d599123ce1a660dcae)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:54.011424 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83030b5-b17f-43c3-92d0-e7f1469ae9c6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:54.140534 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c83030b5-b17f-43c3-92d0-e7f1469ae9c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87a9e1d33b2343d599123ce1a660dcae'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:54.172215 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c83030b5-b17f-43c3-92d0-e7f1469ae9c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87a9e1d33b2343d599123ce1a660dcae'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:54.235549 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c83030b5-b17f-43c3-92d0-e7f1469ae9c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87a9e1d33b2343d599123ce1a660dcae'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:54.237726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c83030b5-b17f-43c3-92d0-e7f1469ae9c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87a9e1d33b2343d599123ce1a660dcae'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:54.279034 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c83030b5-b17f-43c3-92d0-e7f1469ae9c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87a9e1d33b2343d599123ce1a660dcae'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:54.324808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 899/1783] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:14:53 2022] DELETE /identity/v3/users/87a9e1d33b2343d599123ce1a660dcae => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:14:54.326202 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77534212-c309-4530-a24a-e9134247e19a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:54.327005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77534212-c309-4530-a24a-e9134247e19a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:54.327005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77534212-c309-4530-a24a-e9134247e19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:54.347251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba32939-abc5-4534-beb8-65b80623dfbb None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:54.347929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba32939-abc5-4534-beb8-65b80623dfbb None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:54.348456 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba32939-abc5-4534-beb8-65b80623dfbb None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:54.351750 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 900/1784] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 6 16:14:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:14:54.407620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-77534212-c309-4530-a24a-e9134247e19a None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:54.410064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-77534212-c309-4530-a24a-e9134247e19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:54.536741 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a571930-f347-47e4-9dc0-14b683380159 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:54.537289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a571930-f347-47e4-9dc0-14b683380159 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:54.537666 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a571930-f347-47e4-9dc0-14b683380159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:54.581685 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4a571930-f347-47e4-9dc0-14b683380159 None None] MFA Rules not processed for user `6afd4f6ffdae4b16aa5ba757e68c18c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:54.583448 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4a571930-f347-47e4-9dc0-14b683380159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:54.596311 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 885/1785] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:14:54.832349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 901/1786] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 6 16:14:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:14:54.904070 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77534212-c309-4530-a24a-e9134247e19a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:54.907418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-77534212-c309-4530-a24a-e9134247e19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:55.072202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7f45c-9ecb-4ab6-ac83-be4c21abd056 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:55.073094 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7f45c-9ecb-4ab6-ac83-be4c21abd056 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:55.073790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7f45c-9ecb-4ab6-ac83-be4c21abd056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:55.141132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77534212-c309-4530-a24a-e9134247e19a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:55.146738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7d763-6d2e-486c-a76d-47ec347d3dff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:55.147411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7d763-6d2e-486c-a76d-47ec347d3dff service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:55.149555 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7d763-6d2e-486c-a76d-47ec347d3dff service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:55.149881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7d763-6d2e-486c-a76d-47ec347d3dff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:55.172360 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c7d763-6d2e-486c-a76d-47ec347d3dff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:55.182508 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c7d763-6d2e-486c-a76d-47ec347d3dff service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:55.191544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:55.244902 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a4a7f45c-9ecb-4ab6-ac83-be4c21abd056 None None] MFA Rules not processed for user `550277b863fd4797a741b77f716bef56`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:55.247418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a4a7f45c-9ecb-4ab6-ac83-be4c21abd056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:55.253895 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 886/1787] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:55.443640 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7d763-6d2e-486c-a76d-47ec347d3dff service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:55.445587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-59c7d763-6d2e-486c-a76d-47ec347d3dff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:55.446344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 902/1788] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:55 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:55.473928 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7d763-6d2e-486c-a76d-47ec347d3dff service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:55.488244 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b185d86-5748-44c0-9108-c1bf1a26b998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:55.489031 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b185d86-5748-44c0-9108-c1bf1a26b998 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:55.489443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b185d86-5748-44c0-9108-c1bf1a26b998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:55.489819 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b185d86-5748-44c0-9108-c1bf1a26b998 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:55.490610 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b185d86-5748-44c0-9108-c1bf1a26b998 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:55.492831 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b185d86-5748-44c0-9108-c1bf1a26b998 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:55.561175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 887/1789] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:55 2022] POST /identity/v3/projects => generated 403 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:14:55.563944 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7f45c-9ecb-4ab6-ac83-be4c21abd056 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:55.593774 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7f45c-9ecb-4ab6-ac83-be4c21abd056 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:55.678615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b185d86-5748-44c0-9108-c1bf1a26b998 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:55.681307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5b185d86-5748-44c0-9108-c1bf1a26b998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:55.725558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719e42c-8bbb-4b7a-b6d5-227e8750e88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:55.726573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719e42c-8bbb-4b7a-b6d5-227e8750e88a admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:55.727125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719e42c-8bbb-4b7a-b6d5-227e8750e88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:55.727527 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719e42c-8bbb-4b7a-b6d5-227e8750e88a admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:55.728540 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719e42c-8bbb-4b7a-b6d5-227e8750e88a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:55.732277 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719e42c-8bbb-4b7a-b6d5-227e8750e88a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:55.761344 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:55.782074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 903/1790] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:55 2022] POST /identity/v3/users => generated 308 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:55.785175 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719e42c-8bbb-4b7a-b6d5-227e8750e88a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:55.810352 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719e42c-8bbb-4b7a-b6d5-227e8750e88a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:55.823517 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc1dab-abc5-4849-9b75-941042c28112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:55.824185 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc1dab-abc5-4849-9b75-941042c28112 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:55.825096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc1dab-abc5-4849-9b75-941042c28112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:55.825096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc1dab-abc5-4849-9b75-941042c28112 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:55.825437 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edc1dab-abc5-4849-9b75-941042c28112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:55.828615 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edc1dab-abc5-4849-9b75-941042c28112 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:55.841798 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b185d86-5748-44c0-9108-c1bf1a26b998 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:55.843357 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 904/1791] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:55.851453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc1dab-abc5-4849-9b75-941042c28112 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:55.859334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e5140-9767-4ceb-b362-debf795b8d99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:55.860643 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e5140-9767-4ceb-b362-debf795b8d99 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:55.861199 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e5140-9767-4ceb-b362-debf795b8d99 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:55.861919 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e5140-9767-4ceb-b362-debf795b8d99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:55.866279 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc1dab-abc5-4849-9b75-941042c28112 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:55.874352 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ecb8e-1274-479d-8603-ecd0500f7615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:55.875428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ecb8e-1274-479d-8603-ecd0500f7615 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:55.875808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ecb8e-1274-479d-8603-ecd0500f7615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:55.877565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ecb8e-1274-479d-8603-ecd0500f7615 admin admin] PATH_INFO: `/v3/projects/c1fb597484b14cfc93f1a1ee271e89e7/users/c7f67f54867149e6b00f036d65e349d3/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:55.884477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e5140-9767-4ceb-b362-debf795b8d99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:55.888691 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e5140-9767-4ceb-b362-debf795b8d99 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:55.907382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ecb8e-1274-479d-8603-ecd0500f7615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1fb597484b14cfc93f1a1ee271e89e7, user_id=c7f67f54867149e6b00f036d65e349d3, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:55.912531 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ecb8e-1274-479d-8603-ecd0500f7615 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:55.915467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 888/1792] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:14:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:14:55.942257 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 905/1793] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:55 2022] PUT /identity/v3/projects/c1fb597484b14cfc93f1a1ee271e89e7/users/c7f67f54867149e6b00f036d65e349d3/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:55.944370 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e5140-9767-4ceb-b362-debf795b8d99 service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:55.959487 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e5140-9767-4ceb-b362-debf795b8d99 service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:56.036050 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e3dab-4544-4470-a323-da92f66b377b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:56.036775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e3dab-4544-4470-a323-da92f66b377b admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:56.037244 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e3dab-4544-4470-a323-da92f66b377b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:56.037658 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e3dab-4544-4470-a323-da92f66b377b admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:56.038568 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e3dab-4544-4470-a323-da92f66b377b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:56.040397 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e3dab-4544-4470-a323-da92f66b377b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:56.053635 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 889/1794] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:56.058392 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ecb8e-1274-479d-8603-ecd0500f7615 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:56.070396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ecb8e-1274-479d-8603-ecd0500f7615 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:56.075517 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23baf234-38e9-4903-9f47-596ce7ea7fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:56.076403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23baf234-38e9-4903-9f47-596ce7ea7fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:56.076555 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23baf234-38e9-4903-9f47-596ce7ea7fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:56.076780 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23baf234-38e9-4903-9f47-596ce7ea7fd5 admin admin] PATH_INFO: `/v3/projects/c1fb597484b14cfc93f1a1ee271e89e7/users/c7f67f54867149e6b00f036d65e349d3/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:56.078902 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23baf234-38e9-4903-9f47-596ce7ea7fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1fb597484b14cfc93f1a1ee271e89e7, user_id=c7f67f54867149e6b00f036d65e349d3, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:56.082721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23baf234-38e9-4903-9f47-596ce7ea7fd5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:56.095060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 906/1795] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:56 2022] PUT /identity/v3/projects/c1fb597484b14cfc93f1a1ee271e89e7/users/c7f67f54867149e6b00f036d65e349d3/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:56.353605 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e3dab-4544-4470-a323-da92f66b377b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:56.363129 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8d2e3dab-4544-4470-a323-da92f66b377b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:56.390076 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e3dab-4544-4470-a323-da92f66b377b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:56.433641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23baf234-38e9-4903-9f47-596ce7ea7fd5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:56.437882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-23baf234-38e9-4903-9f47-596ce7ea7fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:56.579549 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:56.619296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f52102-adca-4037-b4b6-0ae068c37213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:56.623258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f52102-adca-4037-b4b6-0ae068c37213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:56.623722 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f52102-adca-4037-b4b6-0ae068c37213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:56.624652 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f52102-adca-4037-b4b6-0ae068c37213 admin admin] PATH_INFO: `/v3/projects/4d112432449d40278e0a873ec14bd54f` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:56.626487 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f52102-adca-4037-b4b6-0ae068c37213 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d112432449d40278e0a873ec14bd54f)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:56.630022 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f52102-adca-4037-b4b6-0ae068c37213 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:56.652435 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23baf234-38e9-4903-9f47-596ce7ea7fd5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:56.670412 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e1d55-5a04-435f-a479-833cbb1562db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:56.672139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e1d55-5a04-435f-a479-833cbb1562db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:56.672852 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e1d55-5a04-435f-a479-833cbb1562db admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:56.673827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e1d55-5a04-435f-a479-833cbb1562db admin admin] PATH_INFO: `/v3/projects/0665c54871b848f78e47b948dc00d53c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:56.677321 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e1d55-5a04-435f-a479-833cbb1562db admin admin] RBAC: Authorizing `identity:delete_project(project_id=0665c54871b848f78e47b948dc00d53c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:56.685956 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e1d55-5a04-435f-a479-833cbb1562db admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:56.723770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-23f52102-adca-4037-b4b6-0ae068c37213 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d112432449d40278e0a873ec14bd54f', 'request_id': 'req-23f52102-adca-4037-b4b6-0ae068c37213'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:56.751279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-23f52102-adca-4037-b4b6-0ae068c37213 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d112432449d40278e0a873ec14bd54f', 'request_id': 'req-23f52102-adca-4037-b4b6-0ae068c37213'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:56.761918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-23f52102-adca-4037-b4b6-0ae068c37213 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d112432449d40278e0a873ec14bd54f', 'request_id': 'req-23f52102-adca-4037-b4b6-0ae068c37213'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:56.773050 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:56.802224 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-23f52102-adca-4037-b4b6-0ae068c37213 admin admin] The token cache is being invalidate because project 4d112432449d40278e0a873ec14bd54f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:14:56.803542 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-23f52102-adca-4037-b4b6-0ae068c37213 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:56.806919 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-23f52102-adca-4037-b4b6-0ae068c37213 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:56.816576 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 890/1796] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:14:56 2022] DELETE /identity/v3/projects/4d112432449d40278e0a873ec14bd54f => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:56.823387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee0c3b-626e-4021-88a5-7ced2c0c9450 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:56.823387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee0c3b-626e-4021-88a5-7ced2c0c9450 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:56.824260 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee0c3b-626e-4021-88a5-7ced2c0c9450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:56.895635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0d3e1d55-5a04-435f-a479-833cbb1562db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0665c54871b848f78e47b948dc00d53c', 'request_id': 'req-0d3e1d55-5a04-435f-a479-833cbb1562db'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:56.929526 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-60ee0c3b-626e-4021-88a5-7ced2c0c9450 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:56.933163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-60ee0c3b-626e-4021-88a5-7ced2c0c9450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:56.942963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0d3e1d55-5a04-435f-a479-833cbb1562db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0665c54871b848f78e47b948dc00d53c', 'request_id': 'req-0d3e1d55-5a04-435f-a479-833cbb1562db'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:56.965514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0d3e1d55-5a04-435f-a479-833cbb1562db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0665c54871b848f78e47b948dc00d53c', 'request_id': 'req-0d3e1d55-5a04-435f-a479-833cbb1562db'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:57.023339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0d3e1d55-5a04-435f-a479-833cbb1562db admin admin] The token cache is being invalidate because project 0665c54871b848f78e47b948dc00d53c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:14:57.023815 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0d3e1d55-5a04-435f-a479-833cbb1562db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:57.029538 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0d3e1d55-5a04-435f-a479-833cbb1562db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:57.040153 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 907/1797] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:14:56 2022] DELETE /identity/v3/projects/0665c54871b848f78e47b948dc00d53c => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:57.042706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e1d55-5a04-435f-a479-833cbb1562db admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:57.049968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0d3e1d55-5a04-435f-a479-833cbb1562db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:57.132569 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 891/1798] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:57.147956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8798581-fb52-43ff-ab00-8df0440d1ecb None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:57.148822 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8798581-fb52-43ff-ab00-8df0440d1ecb None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:57.149315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8798581-fb52-43ff-ab00-8df0440d1ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:57.192010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d8798581-fb52-43ff-ab00-8df0440d1ecb None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:57.195355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d8798581-fb52-43ff-ab00-8df0440d1ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:57.218991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 892/1799] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:14:57.222106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8798581-fb52-43ff-ab00-8df0440d1ecb None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:57.243736 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8798581-fb52-43ff-ab00-8df0440d1ecb None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:57.254542 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53793ba-f71c-496f-a69f-76ad97e3c244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:57.255793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53793ba-f71c-496f-a69f-76ad97e3c244 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:57.256294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53793ba-f71c-496f-a69f-76ad97e3c244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:57.256798 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53793ba-f71c-496f-a69f-76ad97e3c244 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:57.258029 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53793ba-f71c-496f-a69f-76ad97e3c244 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:14:57.259431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53793ba-f71c-496f-a69f-76ad97e3c244 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:57.262920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53793ba-f71c-496f-a69f-76ad97e3c244 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:57.278002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e1d55-5a04-435f-a479-833cbb1562db admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:57.291736 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 893/1800] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:14:57.295993 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa0948-6e11-486d-8935-4ee95f1e1aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:57.296787 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53793ba-f71c-496f-a69f-76ad97e3c244 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:57.296960 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa0948-6e11-486d-8935-4ee95f1e1aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:57.297160 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa0948-6e11-486d-8935-4ee95f1e1aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:57.297361 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa0948-6e11-486d-8935-4ee95f1e1aa9 admin admin] PATH_INFO: `/v3/users/9d8c18dae2b14d6c93ddb0ba4d5480d2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:57.299475 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa0948-6e11-486d-8935-4ee95f1e1aa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d8c18dae2b14d6c93ddb0ba4d5480d2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:57.302609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa0948-6e11-486d-8935-4ee95f1e1aa9 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:57.312498 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53793ba-f71c-496f-a69f-76ad97e3c244 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:57.321217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aec46a-cc35-428b-9927-b5a5ff9ef5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:57.322026 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aec46a-cc35-428b-9927-b5a5ff9ef5af admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:57.322455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aec46a-cc35-428b-9927-b5a5ff9ef5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:57.322887 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aec46a-cc35-428b-9927-b5a5ff9ef5af admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:57.323692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aec46a-cc35-428b-9927-b5a5ff9ef5af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:57.328992 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aec46a-cc35-428b-9927-b5a5ff9ef5af admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:57.380077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 894/1801] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:57 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:57.385838 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aec46a-cc35-428b-9927-b5a5ff9ef5af admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:57.394601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6efa0948-6e11-486d-8935-4ee95f1e1aa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d8c18dae2b14d6c93ddb0ba4d5480d2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:57.415286 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6efa0948-6e11-486d-8935-4ee95f1e1aa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d8c18dae2b14d6c93ddb0ba4d5480d2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:57.447443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6efa0948-6e11-486d-8935-4ee95f1e1aa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d8c18dae2b14d6c93ddb0ba4d5480d2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:57.448640 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6efa0948-6e11-486d-8935-4ee95f1e1aa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d8c18dae2b14d6c93ddb0ba4d5480d2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:57.455721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:57.467467 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6efa0948-6e11-486d-8935-4ee95f1e1aa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d8c18dae2b14d6c93ddb0ba4d5480d2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:57.479054 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 908/1802] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:14:57 2022] DELETE /identity/v3/users/9d8c18dae2b14d6c93ddb0ba4d5480d2 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:14:57.513081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aec46a-cc35-428b-9927-b5a5ff9ef5af admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:57.652883 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2fc2f-38c2-49a7-9bda-eff1b6b622ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:57.655517 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2fc2f-38c2-49a7-9bda-eff1b6b622ab admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:57.656081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2fc2f-38c2-49a7-9bda-eff1b6b622ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:57.656831 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2fc2f-38c2-49a7-9bda-eff1b6b622ab admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:57.658170 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2fc2f-38c2-49a7-9bda-eff1b6b622ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:57.661957 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2fc2f-38c2-49a7-9bda-eff1b6b622ab admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:57.679103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:57.710365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 895/1803] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:57 2022] POST /identity/v3/users => generated 293 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:14:57.716627 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa0948-6e11-486d-8935-4ee95f1e1aa9 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:57.719199 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6efa0948-6e11-486d-8935-4ee95f1e1aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:57.753109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa0948-6e11-486d-8935-4ee95f1e1aa9 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:57.766329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230c5a3-0fd9-4029-b31a-6f3d6783bfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:57.767118 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230c5a3-0fd9-4029-b31a-6f3d6783bfaa admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:57.768294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230c5a3-0fd9-4029-b31a-6f3d6783bfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:57.769080 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230c5a3-0fd9-4029-b31a-6f3d6783bfaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:57.770487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0230c5a3-0fd9-4029-b31a-6f3d6783bfaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:57.775031 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0230c5a3-0fd9-4029-b31a-6f3d6783bfaa admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:57.802718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 909/1804] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:57.807419 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2fc2f-38c2-49a7-9bda-eff1b6b622ab admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:57.835407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2fc2f-38c2-49a7-9bda-eff1b6b622ab admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:57.846771 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cdd645-8fcc-45e2-b5ec-56325599e9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:57.848674 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdd645-8fcc-45e2-b5ec-56325599e9df admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:57.849263 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdd645-8fcc-45e2-b5ec-56325599e9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:57.849847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdd645-8fcc-45e2-b5ec-56325599e9df admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:57.851592 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cdd645-8fcc-45e2-b5ec-56325599e9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:57.854190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843b56b-892f-4e8d-8f55-d88fdb36645a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:57.854830 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843b56b-892f-4e8d-8f55-d88fdb36645a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:57.855374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843b56b-892f-4e8d-8f55-d88fdb36645a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:57.871458 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cdd645-8fcc-45e2-b5ec-56325599e9df admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:57.934489 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 896/1805] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:57.943468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cdd645-8fcc-45e2-b5ec-56325599e9df admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:57.948367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-5843b56b-892f-4e8d-8f55-d88fdb36645a None None] MFA Rules not processed for user `c7f67f54867149e6b00f036d65e349d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:57.952434 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5843b56b-892f-4e8d-8f55-d88fdb36645a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:57.995459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cdd645-8fcc-45e2-b5ec-56325599e9df admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:58.024446 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41188e7e-e26c-47b3-912b-5ff9d915193f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:58.026645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41188e7e-e26c-47b3-912b-5ff9d915193f admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:58.032146 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41188e7e-e26c-47b3-912b-5ff9d915193f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:58.032804 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41188e7e-e26c-47b3-912b-5ff9d915193f admin admin] PATH_INFO: `/v3/projects/bfa6e711c01243e0b6c00b4f87da50cf/users/411b7a7b6c6f402e84145f874dd6b5c4/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:58.102342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41188e7e-e26c-47b3-912b-5ff9d915193f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa6e711c01243e0b6c00b4f87da50cf, user_id=411b7a7b6c6f402e84145f874dd6b5c4, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:58.108253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41188e7e-e26c-47b3-912b-5ff9d915193f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:58.165467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 897/1806] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:57 2022] PUT /identity/v3/projects/bfa6e711c01243e0b6c00b4f87da50cf/users/411b7a7b6c6f402e84145f874dd6b5c4/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:58.166778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41188e7e-e26c-47b3-912b-5ff9d915193f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:58.169373 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-41188e7e-e26c-47b3-912b-5ff9d915193f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:58.215017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 910/1807] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:57 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:58.217750 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e146687-17ce-4011-bf6c-8084f1d49564 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:58.219443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e146687-17ce-4011-bf6c-8084f1d49564 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:58.224129 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e146687-17ce-4011-bf6c-8084f1d49564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:58.396148 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8e146687-17ce-4011-bf6c-8084f1d49564 None None] MFA Rules not processed for user `411b7a7b6c6f402e84145f874dd6b5c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:58.402723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8e146687-17ce-4011-bf6c-8084f1d49564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:58.464386 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41188e7e-e26c-47b3-912b-5ff9d915193f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:58.492767 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de4d65-ef72-4e34-8b9f-7189b4b7e4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:58.497579 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de4d65-ef72-4e34-8b9f-7189b4b7e4c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:58.498603 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de4d65-ef72-4e34-8b9f-7189b4b7e4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:58.499923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de4d65-ef72-4e34-8b9f-7189b4b7e4c6 admin admin] PATH_INFO: `/v3/projects/d8cfeaedaf1b4a2ea345294f99260e55` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:58.506716 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de4d65-ef72-4e34-8b9f-7189b4b7e4c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8cfeaedaf1b4a2ea345294f99260e55)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:58.519755 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de4d65-ef72-4e34-8b9f-7189b4b7e4c6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:58.712146 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a6de4d65-ef72-4e34-8b9f-7189b4b7e4c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8cfeaedaf1b4a2ea345294f99260e55', 'request_id': 'req-a6de4d65-ef72-4e34-8b9f-7189b4b7e4c6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:58.724991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 911/1808] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 509 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:58.732311 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e146687-17ce-4011-bf6c-8084f1d49564 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:58.736036 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a6de4d65-ef72-4e34-8b9f-7189b4b7e4c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8cfeaedaf1b4a2ea345294f99260e55', 'request_id': 'req-a6de4d65-ef72-4e34-8b9f-7189b4b7e4c6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:58.754079 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e146687-17ce-4011-bf6c-8084f1d49564 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:58.769383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a6de4d65-ef72-4e34-8b9f-7189b4b7e4c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8cfeaedaf1b4a2ea345294f99260e55', 'request_id': 'req-a6de4d65-ef72-4e34-8b9f-7189b4b7e4c6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:58.790791 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a6de4d65-ef72-4e34-8b9f-7189b4b7e4c6 admin admin] The token cache is being invalidate because project d8cfeaedaf1b4a2ea345294f99260e55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:14:58.791085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a6de4d65-ef72-4e34-8b9f-7189b4b7e4c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:58.792640 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a6de4d65-ef72-4e34-8b9f-7189b4b7e4c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:58.796860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 898/1809] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:14:58 2022] DELETE /identity/v3/projects/d8cfeaedaf1b4a2ea345294f99260e55 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:58.813770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de4d65-ef72-4e34-8b9f-7189b4b7e4c6 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:58.826027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a6de4d65-ef72-4e34-8b9f-7189b4b7e4c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:58.887684 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:58.914353 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ad980-df76-44a7-b898-ce641edd1756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:58.915571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ad980-df76-44a7-b898-ce641edd1756 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:58.916156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ad980-df76-44a7-b898-ce641edd1756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:58.916725 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ad980-df76-44a7-b898-ce641edd1756 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:58.920086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ad980-df76-44a7-b898-ce641edd1756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:58.922520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ad980-df76-44a7-b898-ce641edd1756 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:58.962278 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:59.010993 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 912/1810] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:14:58 2022] POST /identity/v3/projects => generated 383 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:14:59.018305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ad980-df76-44a7-b898-ce641edd1756 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:59.021385 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e48ad980-df76-44a7-b898-ce641edd1756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:59.056479 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de4d65-ef72-4e34-8b9f-7189b4b7e4c6 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:59.189867 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ad980-df76-44a7-b898-ce641edd1756 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:59.198427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f330924-656d-4053-a0aa-f3f5aefc6bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:59.198737 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9e3f7-27d1-45a8-a813-e459e3a6ae7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:59.199349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f330924-656d-4053-a0aa-f3f5aefc6bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:59.199736 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f330924-656d-4053-a0aa-f3f5aefc6bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:59.199926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9e3f7-27d1-45a8-a813-e459e3a6ae7a admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:59.200191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f330924-656d-4053-a0aa-f3f5aefc6bda admin admin] PATH_INFO: `/v3/users/de4e5a300e0d409ba2267389f834bc48` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:59.200607 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9e3f7-27d1-45a8-a813-e459e3a6ae7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:59.201169 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9e3f7-27d1-45a8-a813-e459e3a6ae7a admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:59.201811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f330924-656d-4053-a0aa-f3f5aefc6bda admin admin] RBAC: Authorizing `identity:delete_user(user_id=de4e5a300e0d409ba2267389f834bc48)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:59.203002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9e3f7-27d1-45a8-a813-e459e3a6ae7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:59.204470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f330924-656d-4053-a0aa-f3f5aefc6bda admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:59.207758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9e3f7-27d1-45a8-a813-e459e3a6ae7a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:59.270185 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 913/1811] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:14:59 2022] POST /identity/v3/users => generated 298 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:14:59.271916 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9e3f7-27d1-45a8-a813-e459e3a6ae7a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:59.275183 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ebe9e3f7-27d1-45a8-a813-e459e3a6ae7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:59.300456 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9e3f7-27d1-45a8-a813-e459e3a6ae7a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:59.309523 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166db976-38d0-4b57-8d2e-b9132b6eb99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:59.310634 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166db976-38d0-4b57-8d2e-b9132b6eb99a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:59.311185 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166db976-38d0-4b57-8d2e-b9132b6eb99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:59.311872 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166db976-38d0-4b57-8d2e-b9132b6eb99a admin admin] PATH_INFO: `/v3/projects/b73585d5d3f4491a8197be34e2b758c1` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:59.314255 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166db976-38d0-4b57-8d2e-b9132b6eb99a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b73585d5d3f4491a8197be34e2b758c1)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:59.320031 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166db976-38d0-4b57-8d2e-b9132b6eb99a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:59.325417 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9f330924-656d-4053-a0aa-f3f5aefc6bda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de4e5a300e0d409ba2267389f834bc48'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:59.338426 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9f330924-656d-4053-a0aa-f3f5aefc6bda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de4e5a300e0d409ba2267389f834bc48'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:59.355883 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9f330924-656d-4053-a0aa-f3f5aefc6bda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de4e5a300e0d409ba2267389f834bc48'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:59.356627 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9f330924-656d-4053-a0aa-f3f5aefc6bda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de4e5a300e0d409ba2267389f834bc48'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:59.357464 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9f330924-656d-4053-a0aa-f3f5aefc6bda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de4e5a300e0d409ba2267389f834bc48'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:59.378177 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 899/1812] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:14:58 2022] DELETE /identity/v3/users/de4e5a300e0d409ba2267389f834bc48 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:14:59.380072 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f330924-656d-4053-a0aa-f3f5aefc6bda admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:59.382697 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9f330924-656d-4053-a0aa-f3f5aefc6bda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:59.422015 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-166db976-38d0-4b57-8d2e-b9132b6eb99a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b73585d5d3f4491a8197be34e2b758c1', 'request_id': 'req-166db976-38d0-4b57-8d2e-b9132b6eb99a'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:59.436056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-166db976-38d0-4b57-8d2e-b9132b6eb99a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b73585d5d3f4491a8197be34e2b758c1', 'request_id': 'req-166db976-38d0-4b57-8d2e-b9132b6eb99a'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:59.448154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-166db976-38d0-4b57-8d2e-b9132b6eb99a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b73585d5d3f4491a8197be34e2b758c1', 'request_id': 'req-166db976-38d0-4b57-8d2e-b9132b6eb99a'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:59.477517 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-166db976-38d0-4b57-8d2e-b9132b6eb99a admin admin] The token cache is being invalidate because project b73585d5d3f4491a8197be34e2b758c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:14:59.478029 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-166db976-38d0-4b57-8d2e-b9132b6eb99a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:59.479339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-166db976-38d0-4b57-8d2e-b9132b6eb99a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:14:59.482604 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 914/1813] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:14:59 2022] DELETE /identity/v3/projects/b73585d5d3f4491a8197be34e2b758c1 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:59.493490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb553eb-9d10-4400-84f3-169fc96ff081 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:59.493746 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb553eb-9d10-4400-84f3-169fc96ff081 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:59.494070 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb553eb-9d10-4400-84f3-169fc96ff081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:59.518036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-1bb553eb-9d10-4400-84f3-169fc96ff081 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:59.519640 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1bb553eb-9d10-4400-84f3-169fc96ff081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:59.533807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f330924-656d-4053-a0aa-f3f5aefc6bda admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:59.565334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:59.677179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:14:59.707273 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef0739-517a-4cdb-a6ac-df419f1d511f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:59.708098 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef0739-517a-4cdb-a6ac-df419f1d511f admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:59.708448 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef0739-517a-4cdb-a6ac-df419f1d511f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:59.708810 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef0739-517a-4cdb-a6ac-df419f1d511f admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:59.709555 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef0739-517a-4cdb-a6ac-df419f1d511f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:59.711672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef0739-517a-4cdb-a6ac-df419f1d511f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:59.716896 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 915/1814] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:14:59.729323 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82960b05-7d75-4058-b8bc-328abef640ba None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:59.729837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82960b05-7d75-4058-b8bc-328abef640ba None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:59.730334 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82960b05-7d75-4058-b8bc-328abef640ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:59.741848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 900/1815] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:14:59.749381 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d96fcb-049c-4933-8b81-8cccda69c833 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:59.750308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d96fcb-049c-4933-8b81-8cccda69c833 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:59.751155 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d96fcb-049c-4933-8b81-8cccda69c833 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:59.755973 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 901/1816] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 6 16:14:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 16:14:59.758231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d96fcb-049c-4933-8b81-8cccda69c833 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:14:59.762981 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e4d96fcb-049c-4933-8b81-8cccda69c833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:59.776254 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-82960b05-7d75-4058-b8bc-328abef640ba None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:59.779026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-82960b05-7d75-4058-b8bc-328abef640ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:59.803265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d96fcb-049c-4933-8b81-8cccda69c833 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:14:59.807642 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 916/1817] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:14:59.811161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20da69-d4c9-4d85-8417-1fddbde03568 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:59.811618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20da69-d4c9-4d85-8417-1fddbde03568 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:59.813115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20da69-d4c9-4d85-8417-1fddbde03568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:59.825053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c83df-682a-46ca-890b-c390f0ed1913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:14:59.830641 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c83df-682a-46ca-890b-c390f0ed1913 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:14:59.831548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c83df-682a-46ca-890b-c390f0ed1913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:14:59.832163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c83df-682a-46ca-890b-c390f0ed1913 admin admin] PATH_INFO: `/v3/projects/d16779eda63145f1a34d462a3ba1f2fb/users/69b1568a07744f1187ba3477896865f2/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:14:59.869103 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8e20da69-d4c9-4d85-8417-1fddbde03568 None None] MFA Rules not processed for user `034d75ab747e4c0cab040e47c6a6236e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:14:59.872914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8e20da69-d4c9-4d85-8417-1fddbde03568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:14:59.888732 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c83df-682a-46ca-890b-c390f0ed1913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d16779eda63145f1a34d462a3ba1f2fb, user_id=69b1568a07744f1187ba3477896865f2, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:14:59.895570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c83df-682a-46ca-890b-c390f0ed1913 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:14:59.948446 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 902/1818] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:14:59 2022] PUT /identity/v3/projects/d16779eda63145f1a34d462a3ba1f2fb/users/69b1568a07744f1187ba3477896865f2/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:14:59.956881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c83df-682a-46ca-890b-c390f0ed1913 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:00.061760 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 917/1819] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 6 16:14:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:00.063374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20da69-d4c9-4d85-8417-1fddbde03568 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:00.082147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20da69-d4c9-4d85-8417-1fddbde03568 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:00.135914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c83df-682a-46ca-890b-c390f0ed1913 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:00.147366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbae1b3-8d7a-4526-b5f9-2bb104644371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:00.149248 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbae1b3-8d7a-4526-b5f9-2bb104644371 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:00.149791 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbae1b3-8d7a-4526-b5f9-2bb104644371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:00.150297 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbae1b3-8d7a-4526-b5f9-2bb104644371 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:00.151869 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbae1b3-8d7a-4526-b5f9-2bb104644371 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:00.153359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbae1b3-8d7a-4526-b5f9-2bb104644371 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:00.159701 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbae1b3-8d7a-4526-b5f9-2bb104644371 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:00.171639 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3ec33-f365-4614-a96c-6dcca1b0b405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:00.172634 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3ec33-f365-4614-a96c-6dcca1b0b405 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:00.173119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3ec33-f365-4614-a96c-6dcca1b0b405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:00.173396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3ec33-f365-4614-a96c-6dcca1b0b405 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:00.174635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3ec33-f365-4614-a96c-6dcca1b0b405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:00.176375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3ec33-f365-4614-a96c-6dcca1b0b405 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:00.202919 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 918/1820] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:00.204703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3ec33-f365-4614-a96c-6dcca1b0b405 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:00.208658 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 903/1821] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:00.212330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-74e3ec33-f365-4614-a96c-6dcca1b0b405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:00.216100 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbae1b3-8d7a-4526-b5f9-2bb104644371 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:00.230012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbae1b3-8d7a-4526-b5f9-2bb104644371 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:00.238741 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435e127-fab9-4a18-86af-cfe9c7b09aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:00.239449 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435e127-fab9-4a18-86af-cfe9c7b09aee admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:00.239729 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435e127-fab9-4a18-86af-cfe9c7b09aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:00.240261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435e127-fab9-4a18-86af-cfe9c7b09aee admin admin] PATH_INFO: `/v3/projects/d16779eda63145f1a34d462a3ba1f2fb/users/69b1568a07744f1187ba3477896865f2/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:00.247172 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435e127-fab9-4a18-86af-cfe9c7b09aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d16779eda63145f1a34d462a3ba1f2fb, user_id=69b1568a07744f1187ba3477896865f2, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:00.251534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435e127-fab9-4a18-86af-cfe9c7b09aee admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:00.266977 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 904/1822] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:00 2022] PUT /identity/v3/projects/d16779eda63145f1a34d462a3ba1f2fb/users/69b1568a07744f1187ba3477896865f2/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:00.268766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435e127-fab9-4a18-86af-cfe9c7b09aee admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:00.369741 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3ec33-f365-4614-a96c-6dcca1b0b405 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:00.403650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:00.427680 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435e127-fab9-4a18-86af-cfe9c7b09aee admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:00.435704 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64703da-5b5c-4b1b-ba06-5535c37ed4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:00.436762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64703da-5b5c-4b1b-ba06-5535c37ed4c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:00.437519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64703da-5b5c-4b1b-ba06-5535c37ed4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:00.438345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64703da-5b5c-4b1b-ba06-5535c37ed4c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:00.439357 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64703da-5b5c-4b1b-ba06-5535c37ed4c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:00.444457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64703da-5b5c-4b1b-ba06-5535c37ed4c6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:00.468929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126ca13-07af-46df-bcc6-7d7042c73dd2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034d75ab747e4c0cab040e47c6a6236e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:00.470968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126ca13-07af-46df-bcc6-7d7042c73dd2 service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:00.471679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126ca13-07af-46df-bcc6-7d7042c73dd2 service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:00.472115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126ca13-07af-46df-bcc6-7d7042c73dd2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:00.485703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126ca13-07af-46df-bcc6-7d7042c73dd2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:00.488392 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126ca13-07af-46df-bcc6-7d7042c73dd2 service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:00.490401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:00.504432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 919/1823] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:00.506337 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0b7e1-6c26-45c5-a268-4e8a62170b6c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:00.506991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0b7e1-6c26-45c5-a268-4e8a62170b6c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:00.507445 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0b7e1-6c26-45c5-a268-4e8a62170b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:00.516890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 905/1824] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:00 2022] POST /identity/v3/projects => generated 409 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:00.519512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64703da-5b5c-4b1b-ba06-5535c37ed4c6 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:00.522432 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e64703da-5b5c-4b1b-ba06-5535c37ed4c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:00.557800 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6bf0b7e1-6c26-45c5-a268-4e8a62170b6c None None] MFA Rules not processed for user `69b1568a07744f1187ba3477896865f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:00.567988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6bf0b7e1-6c26-45c5-a268-4e8a62170b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:00.688986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64703da-5b5c-4b1b-ba06-5535c37ed4c6 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:00.699003 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cdbf3-4dec-4003-8554-7bee05b9f21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:00.699683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cdbf3-4dec-4003-8554-7bee05b9f21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:00.699962 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cdbf3-4dec-4003-8554-7bee05b9f21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:00.700237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cdbf3-4dec-4003-8554-7bee05b9f21d admin admin] PATH_INFO: `/v3/projects/120861b4de3a430091fa1b8b0bf913d1` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:00.701472 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cdbf3-4dec-4003-8554-7bee05b9f21d admin admin] RBAC: Authorizing `identity:delete_project(project_id=120861b4de3a430091fa1b8b0bf913d1)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:00.703521 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cdbf3-4dec-4003-8554-7bee05b9f21d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:00.758703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 920/1825] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:00.761133 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0b7e1-6c26-45c5-a268-4e8a62170b6c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:00.785439 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0b7e1-6c26-45c5-a268-4e8a62170b6c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:00.817051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c70cdbf3-4dec-4003-8554-7bee05b9f21d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '120861b4de3a430091fa1b8b0bf913d1', 'request_id': 'req-c70cdbf3-4dec-4003-8554-7bee05b9f21d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:00.836214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c70cdbf3-4dec-4003-8554-7bee05b9f21d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '120861b4de3a430091fa1b8b0bf913d1', 'request_id': 'req-c70cdbf3-4dec-4003-8554-7bee05b9f21d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:00.848486 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c70cdbf3-4dec-4003-8554-7bee05b9f21d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '120861b4de3a430091fa1b8b0bf913d1', 'request_id': 'req-c70cdbf3-4dec-4003-8554-7bee05b9f21d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:00.872737 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c70cdbf3-4dec-4003-8554-7bee05b9f21d admin admin] The token cache is being invalidate because project 120861b4de3a430091fa1b8b0bf913d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:00.873264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c70cdbf3-4dec-4003-8554-7bee05b9f21d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:00.874609 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c70cdbf3-4dec-4003-8554-7bee05b9f21d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:00.878228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 906/1826] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:00 2022] DELETE /identity/v3/projects/120861b4de3a430091fa1b8b0bf913d1 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:00.879664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cdbf3-4dec-4003-8554-7bee05b9f21d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:00.882004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c70cdbf3-4dec-4003-8554-7bee05b9f21d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:00.886119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23cb4f-fd2a-4a67-8f8a-017ec5e19f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:00.887074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23cb4f-fd2a-4a67-8f8a-017ec5e19f0f admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:00.887382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23cb4f-fd2a-4a67-8f8a-017ec5e19f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:00.887662 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23cb4f-fd2a-4a67-8f8a-017ec5e19f0f admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:00.888515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d23cb4f-fd2a-4a67-8f8a-017ec5e19f0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:00.891302 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d23cb4f-fd2a-4a67-8f8a-017ec5e19f0f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:00.932398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 921/1827] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:00 2022] POST /identity/v3/users => generated 305 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:00.933760 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbdf47-8fba-4626-b416-070b5c341d67 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:00.934060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbdf47-8fba-4626-b416-070b5c341d67 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:00.934341 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbdf47-8fba-4626-b416-070b5c341d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:00.959339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-72dbdf47-8fba-4626-b416-070b5c341d67 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:00.960704 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-72dbdf47-8fba-4626-b416-070b5c341d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:00.982219 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 922/1828] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:00.982219 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dbdf47-8fba-4626-b416-070b5c341d67 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:00.982219 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-72dbdf47-8fba-4626-b416-070b5c341d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:00.991646 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dbdf47-8fba-4626-b416-070b5c341d67 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:01.000407 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cdc70-841e-4935-9917-794794ff4957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:01.002838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cdc70-841e-4935-9917-794794ff4957 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:01.003801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cdc70-841e-4935-9917-794794ff4957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:01.004385 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cdc70-841e-4935-9917-794794ff4957 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:01.005326 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cdc70-841e-4935-9917-794794ff4957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:01.008147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cdc70-841e-4935-9917-794794ff4957 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:01.017103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cdbf3-4dec-4003-8554-7bee05b9f21d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:01.023229 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096cc82-596f-4e72-9ca3-80f83f9c9050 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:01.024144 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096cc82-596f-4e72-9ca3-80f83f9c9050 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:01.024518 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096cc82-596f-4e72-9ca3-80f83f9c9050 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:01.024933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096cc82-596f-4e72-9ca3-80f83f9c9050 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:01.026654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 923/1829] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:01.028340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e80498-5eb8-4ad5-927b-20d3419770e9 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:01.028819 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e80498-5eb8-4ad5-927b-20d3419770e9 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:01.029422 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e80498-5eb8-4ad5-927b-20d3419770e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:01.031061 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9096cc82-596f-4e72-9ca3-80f83f9c9050 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:01.056251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c2e80498-5eb8-4ad5-927b-20d3419770e9 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:01.058280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c2e80498-5eb8-4ad5-927b-20d3419770e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:01.071986 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 924/1830] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:01.077106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e80498-5eb8-4ad5-927b-20d3419770e9 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:01.083461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:01.090354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e80498-5eb8-4ad5-927b-20d3419770e9 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:01.100769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a74209-a197-4f34-8d71-8b711bd6c122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:01.102560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a74209-a197-4f34-8d71-8b711bd6c122 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:01.103012 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a74209-a197-4f34-8d71-8b711bd6c122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:01.103502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a74209-a197-4f34-8d71-8b711bd6c122 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:01.105546 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a74209-a197-4f34-8d71-8b711bd6c122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:01.106149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:01.109600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a74209-a197-4f34-8d71-8b711bd6c122 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:01.136749 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 925/1831] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:01.138100 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a74209-a197-4f34-8d71-8b711bd6c122 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:01.144847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096cc82-596f-4e72-9ca3-80f83f9c9050 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:01.148052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096cc82-596f-4e72-9ca3-80f83f9c9050 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:01.165105 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a74209-a197-4f34-8d71-8b711bd6c122 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:01.169854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 907/1832] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:01.171401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096cc82-596f-4e72-9ca3-80f83f9c9050 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:01.173407 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c932b-ac76-4838-8f53-7b999ca31222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:01.174297 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c932b-ac76-4838-8f53-7b999ca31222 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:01.174633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c932b-ac76-4838-8f53-7b999ca31222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:01.174956 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c932b-ac76-4838-8f53-7b999ca31222 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:01.175636 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c932b-ac76-4838-8f53-7b999ca31222 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:01.176135 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c932b-ac76-4838-8f53-7b999ca31222 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:01.178720 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c932b-ac76-4838-8f53-7b999ca31222 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:01.192364 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096cc82-596f-4e72-9ca3-80f83f9c9050 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:01.202848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab5b40-5d41-4475-86ca-0a5937357681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:01.203268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 926/1833] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:01.204279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab5b40-5d41-4475-86ca-0a5937357681 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:01.205384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab5b40-5d41-4475-86ca-0a5937357681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:01.205972 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab5b40-5d41-4475-86ca-0a5937357681 admin admin] PATH_INFO: `/v3/projects/795ca90477604c72912ba07154071959/users/59adfeaccb994120ae348fb7429e2d63/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:01.213327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c932b-ac76-4838-8f53-7b999ca31222 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:01.218293 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2d6c932b-ac76-4838-8f53-7b999ca31222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:01.238568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c932b-ac76-4838-8f53-7b999ca31222 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:01.248257 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b4b47-f45e-4d1f-a694-c41fa7c7d650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:01.249452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b4b47-f45e-4d1f-a694-c41fa7c7d650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:01.250435 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b4b47-f45e-4d1f-a694-c41fa7c7d650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:01.253257 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b4b47-f45e-4d1f-a694-c41fa7c7d650 admin admin] PATH_INFO: `/v3/users/550277b863fd4797a741b77f716bef56` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:01.254020 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b4b47-f45e-4d1f-a694-c41fa7c7d650 admin admin] RBAC: Authorizing `identity:delete_user(user_id=550277b863fd4797a741b77f716bef56)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:01.259572 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b4b47-f45e-4d1f-a694-c41fa7c7d650 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:01.259775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab5b40-5d41-4475-86ca-0a5937357681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=795ca90477604c72912ba07154071959, user_id=59adfeaccb994120ae348fb7429e2d63, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:01.266276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab5b40-5d41-4475-86ca-0a5937357681 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:01.305616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 908/1834] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:01 2022] PUT /identity/v3/projects/795ca90477604c72912ba07154071959/users/59adfeaccb994120ae348fb7429e2d63/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:01.307760 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab5b40-5d41-4475-86ca-0a5937357681 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:01.381169 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-798b4b47-f45e-4d1f-a694-c41fa7c7d650 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '550277b863fd4797a741b77f716bef56'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:01.394140 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-798b4b47-f45e-4d1f-a694-c41fa7c7d650 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '550277b863fd4797a741b77f716bef56'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:01.412820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-798b4b47-f45e-4d1f-a694-c41fa7c7d650 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '550277b863fd4797a741b77f716bef56'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:01.414777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-798b4b47-f45e-4d1f-a694-c41fa7c7d650 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '550277b863fd4797a741b77f716bef56'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:01.429088 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-798b4b47-f45e-4d1f-a694-c41fa7c7d650 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '550277b863fd4797a741b77f716bef56'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:01.437682 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 927/1835] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:01 2022] DELETE /identity/v3/users/550277b863fd4797a741b77f716bef56 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:01.438890 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b4b47-f45e-4d1f-a694-c41fa7c7d650 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:01.441302 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-798b4b47-f45e-4d1f-a694-c41fa7c7d650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:01.484789 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab5b40-5d41-4475-86ca-0a5937357681 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:01.575079 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b4b47-f45e-4d1f-a694-c41fa7c7d650 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:01.582316 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80da63-465e-4224-812e-1ff1cd816f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:01.583000 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80da63-465e-4224-812e-1ff1cd816f18 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:01.583169 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd53af5-4d6a-4fe6-a423-1aa84c7d3657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:01.583417 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80da63-465e-4224-812e-1ff1cd816f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:01.583696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80da63-465e-4224-812e-1ff1cd816f18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:01.585376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80da63-465e-4224-812e-1ff1cd816f18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:01.585585 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd53af5-4d6a-4fe6-a423-1aa84c7d3657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:01.585585 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd53af5-4d6a-4fe6-a423-1aa84c7d3657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:01.586002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd53af5-4d6a-4fe6-a423-1aa84c7d3657 admin admin] PATH_INFO: `/v3/users/411b7a7b6c6f402e84145f874dd6b5c4` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:01.586589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80da63-465e-4224-812e-1ff1cd816f18 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:01.588184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd53af5-4d6a-4fe6-a423-1aa84c7d3657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=411b7a7b6c6f402e84145f874dd6b5c4)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:01.590720 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd53af5-4d6a-4fe6-a423-1aa84c7d3657 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:01.635501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 909/1836] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:01 2022] POST /identity/v3/projects => generated 383 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:01.636943 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a2ba9-990f-41dc-919c-2e6ef854f419 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:01.637234 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a2ba9-990f-41dc-919c-2e6ef854f419 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:01.637507 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a2ba9-990f-41dc-919c-2e6ef854f419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:01.704047 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-335a2ba9-990f-41dc-919c-2e6ef854f419 None None] MFA Rules not processed for user `59adfeaccb994120ae348fb7429e2d63`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:01.706288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-335a2ba9-990f-41dc-919c-2e6ef854f419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:01.711488 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-1dd53af5-4d6a-4fe6-a423-1aa84c7d3657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '411b7a7b6c6f402e84145f874dd6b5c4'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:01.722563 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:01.731086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-1dd53af5-4d6a-4fe6-a423-1aa84c7d3657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '411b7a7b6c6f402e84145f874dd6b5c4'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:01.766866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-1dd53af5-4d6a-4fe6-a423-1aa84c7d3657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '411b7a7b6c6f402e84145f874dd6b5c4'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:01.768936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-1dd53af5-4d6a-4fe6-a423-1aa84c7d3657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '411b7a7b6c6f402e84145f874dd6b5c4'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:01.785029 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-1dd53af5-4d6a-4fe6-a423-1aa84c7d3657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '411b7a7b6c6f402e84145f874dd6b5c4'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:01.798010 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 928/1837] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:01 2022] DELETE /identity/v3/users/411b7a7b6c6f402e84145f874dd6b5c4 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:01.799527 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:01.799901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd53af5-4d6a-4fe6-a423-1aa84c7d3657 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:01.802936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1dd53af5-4d6a-4fe6-a423-1aa84c7d3657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:01.857717 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 910/1838] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:01 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:01.859943 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ec84e-fec7-41ec-9da1-0a296fbce252 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:01.860680 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ec84e-fec7-41ec-9da1-0a296fbce252 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:01.861789 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ec84e-fec7-41ec-9da1-0a296fbce252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:01.889788 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-555ec84e-fec7-41ec-9da1-0a296fbce252 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:01.892798 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-555ec84e-fec7-41ec-9da1-0a296fbce252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:01.996072 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd53af5-4d6a-4fe6-a423-1aa84c7d3657 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:02.005258 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc665447-934b-4cb8-bacb-705439391cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:02.006825 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc665447-934b-4cb8-bacb-705439391cfc admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:02.007399 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc665447-934b-4cb8-bacb-705439391cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:02.008174 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc665447-934b-4cb8-bacb-705439391cfc admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:02.009077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc665447-934b-4cb8-bacb-705439391cfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:02.012741 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc665447-934b-4cb8-bacb-705439391cfc admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:02.053834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 911/1839] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:02.054536 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 929/1840] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:01 2022] POST /identity/v3/users => generated 292 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:02.055238 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ec84e-fec7-41ec-9da1-0a296fbce252 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:02.055755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc665447-934b-4cb8-bacb-705439391cfc admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:02.060851 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-555ec84e-fec7-41ec-9da1-0a296fbce252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:02.061873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cc665447-934b-4cb8-bacb-705439391cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:02.076695 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ec84e-fec7-41ec-9da1-0a296fbce252 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:02.082451 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4837d1c4-bb21-460c-a81e-68b6780b9376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:02.083148 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837d1c4-bb21-460c-a81e-68b6780b9376 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:02.083432 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837d1c4-bb21-460c-a81e-68b6780b9376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:02.083708 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837d1c4-bb21-460c-a81e-68b6780b9376 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:02.084368 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4837d1c4-bb21-460c-a81e-68b6780b9376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:02.086839 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4837d1c4-bb21-460c-a81e-68b6780b9376 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:02.089947 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc665447-934b-4cb8-bacb-705439391cfc admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:02.099070 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6238c7-2380-4a5c-ba84-008be2317a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:02.099944 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6238c7-2380-4a5c-ba84-008be2317a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:02.100347 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6238c7-2380-4a5c-ba84-008be2317a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:02.101368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6238c7-2380-4a5c-ba84-008be2317a34 admin admin] PATH_INFO: `/v3/projects/8368e465a9a148adaf12244d9b96a178` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:02.102838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6238c7-2380-4a5c-ba84-008be2317a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8368e465a9a148adaf12244d9b96a178)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:02.105032 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6238c7-2380-4a5c-ba84-008be2317a34 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:02.128783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 912/1841] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:02 2022] POST /identity/v3/projects => generated 409 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:02.130217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4837d1c4-bb21-460c-a81e-68b6780b9376 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:02.195602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9e6238c7-2380-4a5c-ba84-008be2317a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8368e465a9a148adaf12244d9b96a178', 'request_id': 'req-9e6238c7-2380-4a5c-ba84-008be2317a34'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:02.209624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9e6238c7-2380-4a5c-ba84-008be2317a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8368e465a9a148adaf12244d9b96a178', 'request_id': 'req-9e6238c7-2380-4a5c-ba84-008be2317a34'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:02.220354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9e6238c7-2380-4a5c-ba84-008be2317a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8368e465a9a148adaf12244d9b96a178', 'request_id': 'req-9e6238c7-2380-4a5c-ba84-008be2317a34'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:02.239391 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4837d1c4-bb21-460c-a81e-68b6780b9376 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:02.240312 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9e6238c7-2380-4a5c-ba84-008be2317a34 admin admin] The token cache is being invalidate because project 8368e465a9a148adaf12244d9b96a178 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:02.240714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9e6238c7-2380-4a5c-ba84-008be2317a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:02.241560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9e6238c7-2380-4a5c-ba84-008be2317a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:02.243347 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 930/1842] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:02 2022] DELETE /identity/v3/projects/8368e465a9a148adaf12244d9b96a178 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:02.244254 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6238c7-2380-4a5c-ba84-008be2317a34 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:02.246243 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9e6238c7-2380-4a5c-ba84-008be2317a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:02.320954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:02.334247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea240ae9-807f-45f1-b783-e9073cc619fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:02.334912 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea240ae9-807f-45f1-b783-e9073cc619fe admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:02.335191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea240ae9-807f-45f1-b783-e9073cc619fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:02.335466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea240ae9-807f-45f1-b783-e9073cc619fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:02.336161 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea240ae9-807f-45f1-b783-e9073cc619fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:02.337828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea240ae9-807f-45f1-b783-e9073cc619fe admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:02.338039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6238c7-2380-4a5c-ba84-008be2317a34 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:02.341111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:02.347180 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bbc76c-59c8-4926-8bd4-3969f8d762ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:02.348278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbc76c-59c8-4926-8bd4-3969f8d762ef service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:02.348882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbc76c-59c8-4926-8bd4-3969f8d762ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:02.349254 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbc76c-59c8-4926-8bd4-3969f8d762ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:02.352093 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e8bbc76c-59c8-4926-8bd4-3969f8d762ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:02.352369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 913/1843] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:02.353419 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea240ae9-807f-45f1-b783-e9073cc619fe admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:02.356005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ea240ae9-807f-45f1-b783-e9073cc619fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:02.372137 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea240ae9-807f-45f1-b783-e9073cc619fe admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:02.376161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bbc76c-59c8-4926-8bd4-3969f8d762ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:02.378208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba49c62-0aa0-4be0-9090-b03cb23384d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:02.378909 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bbc76c-59c8-4926-8bd4-3969f8d762ef service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:02.379077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba49c62-0aa0-4be0-9090-b03cb23384d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:02.379227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba49c62-0aa0-4be0-9090-b03cb23384d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:02.379430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba49c62-0aa0-4be0-9090-b03cb23384d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:02.380104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba49c62-0aa0-4be0-9090-b03cb23384d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:02.382132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba49c62-0aa0-4be0-9090-b03cb23384d3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:02.393734 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 931/1844] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:02.395204 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bbc76c-59c8-4926-8bd4-3969f8d762ef service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:02.396690 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e8bbc76c-59c8-4926-8bd4-3969f8d762ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:02.411529 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bbc76c-59c8-4926-8bd4-3969f8d762ef service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:02.415223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 914/1845] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:02 2022] POST /identity/v3/users => generated 311 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:02.416237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba49c62-0aa0-4be0-9090-b03cb23384d3 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:02.418240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fba49c62-0aa0-4be0-9090-b03cb23384d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:02.422217 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b144d-ab78-4db4-9311-5400e14fceb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:02.423053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b144d-ab78-4db4-9311-5400e14fceb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:02.423366 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b144d-ab78-4db4-9311-5400e14fceb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:02.423858 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b144d-ab78-4db4-9311-5400e14fceb3 admin admin] PATH_INFO: `/v3/users/c7f67f54867149e6b00f036d65e349d3` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:02.425492 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b144d-ab78-4db4-9311-5400e14fceb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7f67f54867149e6b00f036d65e349d3)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:02.427779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b144d-ab78-4db4-9311-5400e14fceb3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:02.438806 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba49c62-0aa0-4be0-9090-b03cb23384d3 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:02.448068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd6431-2554-400f-874b-7e1274fbf253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:02.449104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd6431-2554-400f-874b-7e1274fbf253 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:02.449624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd6431-2554-400f-874b-7e1274fbf253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:02.450072 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd6431-2554-400f-874b-7e1274fbf253 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:02.451228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd6431-2554-400f-874b-7e1274fbf253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:02.454533 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd6431-2554-400f-874b-7e1274fbf253 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:02.476344 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 915/1846] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:02.477506 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd6431-2554-400f-874b-7e1274fbf253 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:02.500266 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd6431-2554-400f-874b-7e1274fbf253 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:02.511198 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb2504-2121-4a6a-8e8f-6adbb8c4658b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:02.512538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb2504-2121-4a6a-8e8f-6adbb8c4658b admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:02.513156 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb2504-2121-4a6a-8e8f-6adbb8c4658b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:02.513749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb2504-2121-4a6a-8e8f-6adbb8c4658b admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:02.514998 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb2504-2121-4a6a-8e8f-6adbb8c4658b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:02.518807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb2504-2121-4a6a-8e8f-6adbb8c4658b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:02.527426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c87b144d-ab78-4db4-9311-5400e14fceb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7f67f54867149e6b00f036d65e349d3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:02.540011 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 916/1847] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:02.541613 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb2504-2121-4a6a-8e8f-6adbb8c4658b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:02.548679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c87b144d-ab78-4db4-9311-5400e14fceb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7f67f54867149e6b00f036d65e349d3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:02.571204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb2504-2121-4a6a-8e8f-6adbb8c4658b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:02.582236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c87b144d-ab78-4db4-9311-5400e14fceb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7f67f54867149e6b00f036d65e349d3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:02.585277 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c87b144d-ab78-4db4-9311-5400e14fceb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7f67f54867149e6b00f036d65e349d3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:02.586587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4dba1d-ee35-4140-9e7f-5d711ed64b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:02.587414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dba1d-ee35-4140-9e7f-5d711ed64b3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:02.587827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dba1d-ee35-4140-9e7f-5d711ed64b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:02.591907 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dba1d-ee35-4140-9e7f-5d711ed64b3c admin admin] PATH_INFO: `/v3/projects/1fc4e43b0c1b47beadf4aaf4fe736fd0/users/a2fe2d35d75f4e529eaf6c1290cf3d66/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:02.613940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c87b144d-ab78-4db4-9311-5400e14fceb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7f67f54867149e6b00f036d65e349d3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:02.633935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 932/1848] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:02 2022] DELETE /identity/v3/users/c7f67f54867149e6b00f036d65e349d3 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:02.635666 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b144d-ab78-4db4-9311-5400e14fceb3 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:02.639820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c87b144d-ab78-4db4-9311-5400e14fceb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:02.669686 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4dba1d-ee35-4140-9e7f-5d711ed64b3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fc4e43b0c1b47beadf4aaf4fe736fd0, user_id=a2fe2d35d75f4e529eaf6c1290cf3d66, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:02.676777 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4dba1d-ee35-4140-9e7f-5d711ed64b3c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:02.710458 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 917/1849] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:02 2022] PUT /identity/v3/projects/1fc4e43b0c1b47beadf4aaf4fe736fd0/users/a2fe2d35d75f4e529eaf6c1290cf3d66/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:02.743096 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c432960-4ddc-46db-96d5-00d8b41f619e None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:02.743554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c432960-4ddc-46db-96d5-00d8b41f619e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:02.744069 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c432960-4ddc-46db-96d5-00d8b41f619e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:02.861928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b144d-ab78-4db4-9311-5400e14fceb3 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:02.863865 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-3c432960-4ddc-46db-96d5-00d8b41f619e None None] MFA Rules not processed for user `a2fe2d35d75f4e529eaf6c1290cf3d66`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:02.867125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3c432960-4ddc-46db-96d5-00d8b41f619e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:03.035481 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5036c-df9d-4ffb-9568-817f4fe5c5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:03.040418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5036c-df9d-4ffb-9568-817f4fe5c5d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:03.042613 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5036c-df9d-4ffb-9568-817f4fe5c5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:03.046914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5036c-df9d-4ffb-9568-817f4fe5c5d7 admin admin] PATH_INFO: `/v3/projects/1adc57dadc6743e5ab8f49274878ff2a/users/3732a18629f547efbb85dead7e10cc95/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:03.076018 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:03.087794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 918/1850] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:03.093670 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5036c-df9d-4ffb-9568-817f4fe5c5d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1adc57dadc6743e5ab8f49274878ff2a, user_id=3732a18629f547efbb85dead7e10cc95, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:03.103897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5036c-df9d-4ffb-9568-817f4fe5c5d7 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:03.130065 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c432960-4ddc-46db-96d5-00d8b41f619e None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:03.141241 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3c432960-4ddc-46db-96d5-00d8b41f619e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:03.160088 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 933/1851] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:02 2022] PUT /identity/v3/projects/1adc57dadc6743e5ab8f49274878ff2a/users/3732a18629f547efbb85dead7e10cc95/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:03.169650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5036c-df9d-4ffb-9568-817f4fe5c5d7 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:03.279410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c432960-4ddc-46db-96d5-00d8b41f619e None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:03.344408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:03.355392 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5036c-df9d-4ffb-9568-817f4fe5c5d7 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:03.364882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182668e-9309-43e5-8e10-a6e66fd9d9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:03.365953 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182668e-9309-43e5-8e10-a6e66fd9d9d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:03.366737 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182668e-9309-43e5-8e10-a6e66fd9d9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:03.367651 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182668e-9309-43e5-8e10-a6e66fd9d9d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:03.368446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4182668e-9309-43e5-8e10-a6e66fd9d9d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:03.381253 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4182668e-9309-43e5-8e10-a6e66fd9d9d6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:03.387756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afb45d-ba48-4b3e-beb6-34dfc8adc2fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:03.389510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afb45d-ba48-4b3e-beb6-34dfc8adc2fd service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:03.390290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afb45d-ba48-4b3e-beb6-34dfc8adc2fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:03.391076 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afb45d-ba48-4b3e-beb6-34dfc8adc2fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:03.420159 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 934/1852] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:03.421879 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182668e-9309-43e5-8e10-a6e66fd9d9d6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:03.427489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4182668e-9309-43e5-8e10-a6e66fd9d9d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:03.442210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182668e-9309-43e5-8e10-a6e66fd9d9d6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:03.447339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc5d0c-6073-404b-851d-95fa0e2d273d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:03.448030 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc5d0c-6073-404b-851d-95fa0e2d273d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:03.448350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc5d0c-6073-404b-851d-95fa0e2d273d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:03.448607 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc5d0c-6073-404b-851d-95fa0e2d273d admin admin] PATH_INFO: `/v3/projects/bfa6e711c01243e0b6c00b4f87da50cf` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:03.450024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc5d0c-6073-404b-851d-95fa0e2d273d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfa6e711c01243e0b6c00b4f87da50cf)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:03.452206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc5d0c-6073-404b-851d-95fa0e2d273d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:03.511633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afb45d-ba48-4b3e-beb6-34dfc8adc2fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:03.514313 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afb45d-ba48-4b3e-beb6-34dfc8adc2fd service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:03.521306 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3ffc5d0c-6073-404b-851d-95fa0e2d273d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfa6e711c01243e0b6c00b4f87da50cf', 'request_id': 'req-3ffc5d0c-6073-404b-851d-95fa0e2d273d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:03.532060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 919/1853] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:03.533366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afb45d-ba48-4b3e-beb6-34dfc8adc2fd service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:03.534373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3ffc5d0c-6073-404b-851d-95fa0e2d273d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfa6e711c01243e0b6c00b4f87da50cf', 'request_id': 'req-3ffc5d0c-6073-404b-851d-95fa0e2d273d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:03.535552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-07afb45d-ba48-4b3e-beb6-34dfc8adc2fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:03.546842 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3ffc5d0c-6073-404b-851d-95fa0e2d273d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfa6e711c01243e0b6c00b4f87da50cf', 'request_id': 'req-3ffc5d0c-6073-404b-851d-95fa0e2d273d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:03.558031 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afb45d-ba48-4b3e-beb6-34dfc8adc2fd service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:03.575996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3ffc5d0c-6073-404b-851d-95fa0e2d273d admin admin] The token cache is being invalidate because project bfa6e711c01243e0b6c00b4f87da50cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:03.576511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3ffc5d0c-6073-404b-851d-95fa0e2d273d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:03.577683 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3ffc5d0c-6073-404b-851d-95fa0e2d273d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:03.581426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 935/1854] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:03 2022] DELETE /identity/v3/projects/bfa6e711c01243e0b6c00b4f87da50cf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:03.582873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc5d0c-6073-404b-851d-95fa0e2d273d admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:03.585150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3ffc5d0c-6073-404b-851d-95fa0e2d273d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:03.656318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fbd7a-24b7-40cb-9e65-449e160b3b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:03.657215 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fbd7a-24b7-40cb-9e65-449e160b3b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:03.657591 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fbd7a-24b7-40cb-9e65-449e160b3b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:03.657978 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fbd7a-24b7-40cb-9e65-449e160b3b61 admin admin] PATH_INFO: `/v3/projects/c1fb597484b14cfc93f1a1ee271e89e7` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:03.660122 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fbd7a-24b7-40cb-9e65-449e160b3b61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1fb597484b14cfc93f1a1ee271e89e7)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:03.662637 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fbd7a-24b7-40cb-9e65-449e160b3b61 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:03.699938 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc5d0c-6073-404b-851d-95fa0e2d273d admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:03.707571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf0520-7403-482c-9c6e-f94adfc2b33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:03.708533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf0520-7403-482c-9c6e-f94adfc2b33b admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:03.708940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf0520-7403-482c-9c6e-f94adfc2b33b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:03.709544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf0520-7403-482c-9c6e-f94adfc2b33b admin admin] PATH_INFO: `/v3/projects/1adc57dadc6743e5ab8f49274878ff2a/users/3732a18629f547efbb85dead7e10cc95/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:03.712023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf0520-7403-482c-9c6e-f94adfc2b33b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1adc57dadc6743e5ab8f49274878ff2a, user_id=3732a18629f547efbb85dead7e10cc95, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:03.716069 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf0520-7403-482c-9c6e-f94adfc2b33b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:03.725039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:03.732873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 936/1855] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:03 2022] PUT /identity/v3/projects/1adc57dadc6743e5ab8f49274878ff2a/users/3732a18629f547efbb85dead7e10cc95/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:03.733612 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf0520-7403-482c-9c6e-f94adfc2b33b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:03.738967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ddaf0520-7403-482c-9c6e-f94adfc2b33b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:03.753226 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-761fbd7a-24b7-40cb-9e65-449e160b3b61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1fb597484b14cfc93f1a1ee271e89e7', 'request_id': 'req-761fbd7a-24b7-40cb-9e65-449e160b3b61'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:03.764849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-761fbd7a-24b7-40cb-9e65-449e160b3b61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1fb597484b14cfc93f1a1ee271e89e7', 'request_id': 'req-761fbd7a-24b7-40cb-9e65-449e160b3b61'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:03.774282 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-761fbd7a-24b7-40cb-9e65-449e160b3b61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1fb597484b14cfc93f1a1ee271e89e7', 'request_id': 'req-761fbd7a-24b7-40cb-9e65-449e160b3b61'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:03.790282 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-761fbd7a-24b7-40cb-9e65-449e160b3b61 admin admin] The token cache is being invalidate because project c1fb597484b14cfc93f1a1ee271e89e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:03.790555 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-761fbd7a-24b7-40cb-9e65-449e160b3b61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:03.791389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-761fbd7a-24b7-40cb-9e65-449e160b3b61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:03.793267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 920/1856] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:03 2022] DELETE /identity/v3/projects/c1fb597484b14cfc93f1a1ee271e89e7 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:03.794468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fbd7a-24b7-40cb-9e65-449e160b3b61 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:03.796380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-761fbd7a-24b7-40cb-9e65-449e160b3b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:03.836473 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf0520-7403-482c-9c6e-f94adfc2b33b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:03.840624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:03.843175 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf01ce-2270-4a2a-a9e3-72bb30aae823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:03.844056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf01ce-2270-4a2a-a9e3-72bb30aae823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:03.844405 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf01ce-2270-4a2a-a9e3-72bb30aae823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:03.844823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf01ce-2270-4a2a-a9e3-72bb30aae823 admin admin] PATH_INFO: `/v3/users/a2fe2d35d75f4e529eaf6c1290cf3d66` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:03.846503 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf01ce-2270-4a2a-a9e3-72bb30aae823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2fe2d35d75f4e529eaf6c1290cf3d66)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:03.849635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf01ce-2270-4a2a-a9e3-72bb30aae823 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:03.901828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fbd7a-24b7-40cb-9e65-449e160b3b61 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:03.908583 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478024da-449c-41d6-be52-0ddcc7dc18b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:03.909264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478024da-449c-41d6-be52-0ddcc7dc18b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:03.909582 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478024da-449c-41d6-be52-0ddcc7dc18b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:03.909855 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478024da-449c-41d6-be52-0ddcc7dc18b5 admin admin] PATH_INFO: `/v3/users/69b1568a07744f1187ba3477896865f2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:03.911190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478024da-449c-41d6-be52-0ddcc7dc18b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69b1568a07744f1187ba3477896865f2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:03.914968 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478024da-449c-41d6-be52-0ddcc7dc18b5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:03.928509 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-debf01ce-2270-4a2a-a9e3-72bb30aae823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2fe2d35d75f4e529eaf6c1290cf3d66'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:03.940203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-debf01ce-2270-4a2a-a9e3-72bb30aae823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2fe2d35d75f4e529eaf6c1290cf3d66'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:03.960279 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-debf01ce-2270-4a2a-a9e3-72bb30aae823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2fe2d35d75f4e529eaf6c1290cf3d66'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:03.961366 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-debf01ce-2270-4a2a-a9e3-72bb30aae823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2fe2d35d75f4e529eaf6c1290cf3d66'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:03.974319 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-debf01ce-2270-4a2a-a9e3-72bb30aae823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2fe2d35d75f4e529eaf6c1290cf3d66'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:03.986219 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 937/1857] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:03 2022] DELETE /identity/v3/users/a2fe2d35d75f4e529eaf6c1290cf3d66 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:03.987369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-478024da-449c-41d6-be52-0ddcc7dc18b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69b1568a07744f1187ba3477896865f2'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:03.993056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb11e5-a726-445b-a2be-26f820ebd28d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:03.993482 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb11e5-a726-445b-a2be-26f820ebd28d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:03.994060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb11e5-a726-445b-a2be-26f820ebd28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:04.000569 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-478024da-449c-41d6-be52-0ddcc7dc18b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69b1568a07744f1187ba3477896865f2'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:04.023035 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-478024da-449c-41d6-be52-0ddcc7dc18b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69b1568a07744f1187ba3477896865f2'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:04.024044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-478024da-449c-41d6-be52-0ddcc7dc18b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69b1568a07744f1187ba3477896865f2'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:04.025042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-478024da-449c-41d6-be52-0ddcc7dc18b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69b1568a07744f1187ba3477896865f2'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:04.040191 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2fbb11e5-a726-445b-a2be-26f820ebd28d None None] MFA Rules not processed for user `3732a18629f547efbb85dead7e10cc95`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:04.042500 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2fbb11e5-a726-445b-a2be-26f820ebd28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:04.046024 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 921/1858] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:03 2022] DELETE /identity/v3/users/69b1568a07744f1187ba3477896865f2 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:04.047434 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94550c5a-cc0b-4ffe-93ee-f6b625fb8dd2 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:04.047731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94550c5a-cc0b-4ffe-93ee-f6b625fb8dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:04.048501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94550c5a-cc0b-4ffe-93ee-f6b625fb8dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:04.073944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-94550c5a-cc0b-4ffe-93ee-f6b625fb8dd2 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:04.075768 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-94550c5a-cc0b-4ffe-93ee-f6b625fb8dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:04.197046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 922/1859] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:04.197046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12712368-fee8-44d8-b699-a8052ed7ecd7 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:04.197046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12712368-fee8-44d8-b699-a8052ed7ecd7 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:04.197046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12712368-fee8-44d8-b699-a8052ed7ecd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:04.208938 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 938/1860] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:03 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:04.211317 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d05b3a-fc74-4def-8ca1-f08ef2cef62a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:04.211811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d05b3a-fc74-4def-8ca1-f08ef2cef62a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:04.212300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d05b3a-fc74-4def-8ca1-f08ef2cef62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:04.228768 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:04.260421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-12712368-fee8-44d8-b699-a8052ed7ecd7 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:04.271240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-12712368-fee8-44d8-b699-a8052ed7ecd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:04.278758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a8d05b3a-fc74-4def-8ca1-f08ef2cef62a None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:04.281413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a8d05b3a-fc74-4def-8ca1-f08ef2cef62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:04.287696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 923/1861] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:04.289282 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12712368-fee8-44d8-b699-a8052ed7ecd7 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:04.294722 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-12712368-fee8-44d8-b699-a8052ed7ecd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:04.307323 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 939/1862] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:04.314272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d05b3a-fc74-4def-8ca1-f08ef2cef62a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:04.335628 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d05b3a-fc74-4def-8ca1-f08ef2cef62a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:04.343265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e870686-12d5-4d62-9176-1561b068ad35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:04.344165 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e870686-12d5-4d62-9176-1561b068ad35 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:04.344715 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e870686-12d5-4d62-9176-1561b068ad35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:04.345237 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e870686-12d5-4d62-9176-1561b068ad35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:04.347486 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e870686-12d5-4d62-9176-1561b068ad35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:04.348005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e870686-12d5-4d62-9176-1561b068ad35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:04.352675 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e870686-12d5-4d62-9176-1561b068ad35 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:04.374587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 940/1863] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:04.376407 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e870686-12d5-4d62-9176-1561b068ad35 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:04.379378 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6e870686-12d5-4d62-9176-1561b068ad35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:04.398239 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:04.436067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12712368-fee8-44d8-b699-a8052ed7ecd7 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:04.442075 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a11a9-fa50-4a11-9130-aee77add3b01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:04.442746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a11a9-fa50-4a11-9130-aee77add3b01 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:04.443505 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a11a9-fa50-4a11-9130-aee77add3b01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:04.443798 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a11a9-fa50-4a11-9130-aee77add3b01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:04.457001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a11a9-fa50-4a11-9130-aee77add3b01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:04.458508 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a11a9-fa50-4a11-9130-aee77add3b01 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:04.472944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 924/1864] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:04 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:04.474297 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a11a9-fa50-4a11-9130-aee77add3b01 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:04.487836 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a11a9-fa50-4a11-9130-aee77add3b01 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:04.495959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04420344-078f-4225-b13c-948201373802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:04.496707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04420344-078f-4225-b13c-948201373802 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:04.497296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04420344-078f-4225-b13c-948201373802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:04.497711 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04420344-078f-4225-b13c-948201373802 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:04.498547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04420344-078f-4225-b13c-948201373802 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:04.502586 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04420344-078f-4225-b13c-948201373802 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:04.605376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 925/1865] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:04 2022] POST /identity/v3/projects => generated 355 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:04.613953 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04420344-078f-4225-b13c-948201373802 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:04.615139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e870686-12d5-4d62-9176-1561b068ad35 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:04.751559 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e852c-167d-4b24-9d49-56df552dfc3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:04.753745 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e852c-167d-4b24-9d49-56df552dfc3e service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:04.754416 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e852c-167d-4b24-9d49-56df552dfc3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:04.755044 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e852c-167d-4b24-9d49-56df552dfc3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:04.828356 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04420344-078f-4225-b13c-948201373802 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:04.837442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5daa3f-a249-49f6-b1a0-d9985bf3123b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:04.838085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5daa3f-a249-49f6-b1a0-d9985bf3123b admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:04.838567 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5daa3f-a249-49f6-b1a0-d9985bf3123b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:04.839456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5daa3f-a249-49f6-b1a0-d9985bf3123b admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:04.840158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5daa3f-a249-49f6-b1a0-d9985bf3123b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:04.843818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5daa3f-a249-49f6-b1a0-d9985bf3123b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:04.901234 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 926/1866] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:04 2022] POST /identity/v3/users => generated 278 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:15:04.907626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5daa3f-a249-49f6-b1a0-d9985bf3123b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:04.926817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5daa3f-a249-49f6-b1a0-d9985bf3123b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:04.933015 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e852c-167d-4b24-9d49-56df552dfc3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:04.933730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15d0be-31a4-402a-8976-0e66bc3f9271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:04.934363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15d0be-31a4-402a-8976-0e66bc3f9271 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:04.934646 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15d0be-31a4-402a-8976-0e66bc3f9271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:04.935616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15d0be-31a4-402a-8976-0e66bc3f9271 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:04.936382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15d0be-31a4-402a-8976-0e66bc3f9271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:04.937410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e852c-167d-4b24-9d49-56df552dfc3e service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:04.952436 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15d0be-31a4-402a-8976-0e66bc3f9271 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:04.960365 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 941/1867] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:04.961953 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e852c-167d-4b24-9d49-56df552dfc3e service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:04.964377 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-842e852c-167d-4b24-9d49-56df552dfc3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:04.993308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e852c-167d-4b24-9d49-56df552dfc3e service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:04.999889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f82794-133a-4e63-aba5-38bf9b72f0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:05.000611 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f82794-133a-4e63-aba5-38bf9b72f0cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:05.000958 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f82794-133a-4e63-aba5-38bf9b72f0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:05.001465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f82794-133a-4e63-aba5-38bf9b72f0cb admin admin] PATH_INFO: `/v3/projects/d16779eda63145f1a34d462a3ba1f2fb` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:05.003532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f82794-133a-4e63-aba5-38bf9b72f0cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d16779eda63145f1a34d462a3ba1f2fb)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:05.006113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f82794-133a-4e63-aba5-38bf9b72f0cb admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:05.011080 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 927/1868] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:05.012478 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15d0be-31a4-402a-8976-0e66bc3f9271 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:05.084032 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:05.092109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-24f82794-133a-4e63-aba5-38bf9b72f0cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd16779eda63145f1a34d462a3ba1f2fb', 'request_id': 'req-24f82794-133a-4e63-aba5-38bf9b72f0cb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:05.101823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-24f82794-133a-4e63-aba5-38bf9b72f0cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd16779eda63145f1a34d462a3ba1f2fb', 'request_id': 'req-24f82794-133a-4e63-aba5-38bf9b72f0cb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:05.108632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-24f82794-133a-4e63-aba5-38bf9b72f0cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd16779eda63145f1a34d462a3ba1f2fb', 'request_id': 'req-24f82794-133a-4e63-aba5-38bf9b72f0cb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:05.112216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15d0be-31a4-402a-8976-0e66bc3f9271 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:05.128996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-24f82794-133a-4e63-aba5-38bf9b72f0cb admin admin] The token cache is being invalidate because project d16779eda63145f1a34d462a3ba1f2fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:05.129556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-24f82794-133a-4e63-aba5-38bf9b72f0cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:05.131055 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-24f82794-133a-4e63-aba5-38bf9b72f0cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:05.132986 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 942/1869] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:04 2022] DELETE /identity/v3/projects/d16779eda63145f1a34d462a3ba1f2fb => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:05.134825 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f82794-133a-4e63-aba5-38bf9b72f0cb admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:05.139032 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-24f82794-133a-4e63-aba5-38bf9b72f0cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:05.206967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2bb84-0f9f-4f2d-b0fd-261d9a0a49ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:05.208786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2bb84-0f9f-4f2d-b0fd-261d9a0a49ed service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:05.209282 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2bb84-0f9f-4f2d-b0fd-261d9a0a49ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:05.209780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2bb84-0f9f-4f2d-b0fd-261d9a0a49ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:05.215370 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5dc2bb84-0f9f-4f2d-b0fd-261d9a0a49ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:05.226190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:05.260449 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f82794-133a-4e63-aba5-38bf9b72f0cb admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:05.265750 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de808ff-3c95-430d-b59a-55fdcec47a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:05.266372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de808ff-3c95-430d-b59a-55fdcec47a81 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:05.266643 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de808ff-3c95-430d-b59a-55fdcec47a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:05.266942 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de808ff-3c95-430d-b59a-55fdcec47a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:05.268365 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de808ff-3c95-430d-b59a-55fdcec47a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:05.270294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de808ff-3c95-430d-b59a-55fdcec47a81 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:05.282718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 943/1870] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:05.284721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ed817-874b-4160-bfdc-f3fdc9f859a7 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:05.285007 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ed817-874b-4160-bfdc-f3fdc9f859a7 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:05.285282 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ed817-874b-4160-bfdc-f3fdc9f859a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:05.308203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-be8ed817-874b-4160-bfdc-f3fdc9f859a7 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:05.309903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-be8ed817-874b-4160-bfdc-f3fdc9f859a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:05.323284 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 944/1871] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:05.324796 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ed817-874b-4160-bfdc-f3fdc9f859a7 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:05.335240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2bb84-0f9f-4f2d-b0fd-261d9a0a49ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:05.340128 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2bb84-0f9f-4f2d-b0fd-261d9a0a49ed service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:05.344616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ed817-874b-4160-bfdc-f3fdc9f859a7 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:05.359290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e87b8-602d-41e8-ae8f-1020970648ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:05.360200 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e87b8-602d-41e8-ae8f-1020970648ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:05.362216 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e87b8-602d-41e8-ae8f-1020970648ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:05.362216 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e87b8-602d-41e8-ae8f-1020970648ae admin admin] PATH_INFO: `/v3/projects/5b7b1d0918ee42b8b2d779f84874ea36/users/fbf321d4bfbc4f248c79ac584554c00b/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:05.363672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 928/1872] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:05 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:05.366065 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed61f7d-dc0a-4845-bfb4-fd65844be2e8 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:05.366929 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed61f7d-dc0a-4845-bfb4-fd65844be2e8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:05.367613 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed61f7d-dc0a-4845-bfb4-fd65844be2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:05.395706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e87b8-602d-41e8-ae8f-1020970648ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7b1d0918ee42b8b2d779f84874ea36, user_id=fbf321d4bfbc4f248c79ac584554c00b, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:05.399443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e87b8-602d-41e8-ae8f-1020970648ae admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:05.411207 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-5ed61f7d-dc0a-4845-bfb4-fd65844be2e8 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:05.413302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5ed61f7d-dc0a-4845-bfb4-fd65844be2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:05.438965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 945/1873] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:05 2022] PUT /identity/v3/projects/5b7b1d0918ee42b8b2d779f84874ea36/users/fbf321d4bfbc4f248c79ac584554c00b/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:05.442197 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e87b8-602d-41e8-ae8f-1020970648ae admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:05.445866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-556e87b8-602d-41e8-ae8f-1020970648ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:05.535632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 929/1874] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:05.538138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10e385-5d34-48c9-be74-29a248751694 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:05.538872 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10e385-5d34-48c9-be74-29a248751694 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:05.539363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10e385-5d34-48c9-be74-29a248751694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:05.613848 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e87b8-602d-41e8-ae8f-1020970648ae admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:05.639663 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-df10e385-5d34-48c9-be74-29a248751694 None None] MFA Rules not processed for user `fbf321d4bfbc4f248c79ac584554c00b`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:05.643501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-df10e385-5d34-48c9-be74-29a248751694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:05.648714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f7c64-25e7-4c73-b42d-0672dc0e8d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:05.651840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f7c64-25e7-4c73-b42d-0672dc0e8d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:05.655443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f7c64-25e7-4c73-b42d-0672dc0e8d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:05.656394 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f7c64-25e7-4c73-b42d-0672dc0e8d44 admin admin] PATH_INFO: `/v3/projects/1fc4e43b0c1b47beadf4aaf4fe736fd0` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:05.676428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f7c64-25e7-4c73-b42d-0672dc0e8d44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fc4e43b0c1b47beadf4aaf4fe736fd0)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:05.698261 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f7c64-25e7-4c73-b42d-0672dc0e8d44 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:05.775775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 930/1875] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:05.776973 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10e385-5d34-48c9-be74-29a248751694 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:05.821301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10e385-5d34-48c9-be74-29a248751694 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:05.835296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b5036-0a20-47c6-aa16-8a5c6d112e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:05.838032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b5036-0a20-47c6-aa16-8a5c6d112e81 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:05.839529 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b5036-0a20-47c6-aa16-8a5c6d112e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:05.840709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b5036-0a20-47c6-aa16-8a5c6d112e81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:05.844914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b5036-0a20-47c6-aa16-8a5c6d112e81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:05.845418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b5036-0a20-47c6-aa16-8a5c6d112e81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:05.848359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b5036-0a20-47c6-aa16-8a5c6d112e81 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:05.872214 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:05.902314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-867f7c64-25e7-4c73-b42d-0672dc0e8d44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fc4e43b0c1b47beadf4aaf4fe736fd0', 'request_id': 'req-867f7c64-25e7-4c73-b42d-0672dc0e8d44'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:05.930740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 931/1876] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:05.934268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b5036-0a20-47c6-aa16-8a5c6d112e81 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:05.942729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-867f7c64-25e7-4c73-b42d-0672dc0e8d44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fc4e43b0c1b47beadf4aaf4fe736fd0', 'request_id': 'req-867f7c64-25e7-4c73-b42d-0672dc0e8d44'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:05.968713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-867f7c64-25e7-4c73-b42d-0672dc0e8d44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fc4e43b0c1b47beadf4aaf4fe736fd0', 'request_id': 'req-867f7c64-25e7-4c73-b42d-0672dc0e8d44'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:05.988241 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b5036-0a20-47c6-aa16-8a5c6d112e81 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:05.995605 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-867f7c64-25e7-4c73-b42d-0672dc0e8d44 admin admin] The token cache is being invalidate because project 1fc4e43b0c1b47beadf4aaf4fe736fd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:05.996410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-867f7c64-25e7-4c73-b42d-0672dc0e8d44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:05.997666 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-867f7c64-25e7-4c73-b42d-0672dc0e8d44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:06.000058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 946/1877] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:05 2022] DELETE /identity/v3/projects/1fc4e43b0c1b47beadf4aaf4fe736fd0 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:06.001490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f7c64-25e7-4c73-b42d-0672dc0e8d44 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:06.004480 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-867f7c64-25e7-4c73-b42d-0672dc0e8d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:06.036936 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:06.135875 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f7c64-25e7-4c73-b42d-0672dc0e8d44 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:06.143617 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7792b-52f2-4adf-bcd4-a3ee74a60bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:06.144711 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7792b-52f2-4adf-bcd4-a3ee74a60bfa admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:06.145223 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7792b-52f2-4adf-bcd4-a3ee74a60bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:06.145698 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7792b-52f2-4adf-bcd4-a3ee74a60bfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:06.146518 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7792b-52f2-4adf-bcd4-a3ee74a60bfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:06.162962 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7792b-52f2-4adf-bcd4-a3ee74a60bfa admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:06.201051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7e721-fc55-4c6f-a33d-80ff14b17271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:06.202396 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7e721-fc55-4c6f-a33d-80ff14b17271 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:06.203108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7e721-fc55-4c6f-a33d-80ff14b17271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:06.203845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7e721-fc55-4c6f-a33d-80ff14b17271 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:06.205981 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7e721-fc55-4c6f-a33d-80ff14b17271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:06.211568 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7e721-fc55-4c6f-a33d-80ff14b17271 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:06.294614 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 932/1878] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:05 2022] POST /identity/v3/projects => generated 357 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:06.302815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e005c8-c459-467b-b47b-7ad5f0f37314 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:06.303495 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e005c8-c459-467b-b47b-7ad5f0f37314 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:06.303937 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e005c8-c459-467b-b47b-7ad5f0f37314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:06.304264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 947/1879] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:06 2022] POST /identity/v3/projects => generated 385 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:06.305696 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7792b-52f2-4adf-bcd4-a3ee74a60bfa admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:06.309380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-07f7792b-52f2-4adf-bcd4-a3ee74a60bfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:06.347389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-a6e005c8-c459-467b-b47b-7ad5f0f37314 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:06.351407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a6e005c8-c459-467b-b47b-7ad5f0f37314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:06.491846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7792b-52f2-4adf-bcd4-a3ee74a60bfa admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:06.495949 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 933/1880] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:06.498384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e005c8-c459-467b-b47b-7ad5f0f37314 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:06.503452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0614237-963b-4887-9ece-f25d507f1b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:06.504773 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0614237-963b-4887-9ece-f25d507f1b9c admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:06.505391 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0614237-963b-4887-9ece-f25d507f1b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:06.506386 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0614237-963b-4887-9ece-f25d507f1b9c admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:06.508248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0614237-963b-4887-9ece-f25d507f1b9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:06.512913 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0614237-963b-4887-9ece-f25d507f1b9c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:06.521726 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e005c8-c459-467b-b47b-7ad5f0f37314 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:06.528282 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6143850-9764-413c-badc-d45708e2efa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:06.529675 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6143850-9764-413c-badc-d45708e2efa1 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:06.530028 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6143850-9764-413c-badc-d45708e2efa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:06.530305 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6143850-9764-413c-badc-d45708e2efa1 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:06.531004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6143850-9764-413c-badc-d45708e2efa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:06.534210 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6143850-9764-413c-badc-d45708e2efa1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:06.574943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 948/1881] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:06 2022] POST /identity/v3/users => generated 285 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:06.576966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1c396-e88e-4137-b53f-e5a3f2331d36 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:06.577479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1c396-e88e-4137-b53f-e5a3f2331d36 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:06.578118 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1c396-e88e-4137-b53f-e5a3f2331d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:06.578998 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 934/1882] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:06 2022] POST /identity/v3/users => generated 293 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:06.581061 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6143850-9764-413c-badc-d45708e2efa1 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:06.602509 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6143850-9764-413c-badc-d45708e2efa1 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:06.608940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea74844-190f-4e42-bd5c-dfebf9e1743b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:06.609575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea74844-190f-4e42-bd5c-dfebf9e1743b admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:06.609876 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea74844-190f-4e42-bd5c-dfebf9e1743b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:06.610210 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea74844-190f-4e42-bd5c-dfebf9e1743b admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:06.611102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea74844-190f-4e42-bd5c-dfebf9e1743b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:06.612834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea74844-190f-4e42-bd5c-dfebf9e1743b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:06.613047 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f8b1c396-e88e-4137-b53f-e5a3f2331d36 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:06.614724 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f8b1c396-e88e-4137-b53f-e5a3f2331d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:06.624757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 949/1883] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:06.625050 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 935/1884] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:06.625642 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1c396-e88e-4137-b53f-e5a3f2331d36 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:06.631161 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea74844-190f-4e42-bd5c-dfebf9e1743b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:06.637165 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1c396-e88e-4137-b53f-e5a3f2331d36 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:06.642828 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077da5ec-8318-401c-b98d-7a9357ca345e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:06.643638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077da5ec-8318-401c-b98d-7a9357ca345e admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:06.643954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077da5ec-8318-401c-b98d-7a9357ca345e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:06.644232 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077da5ec-8318-401c-b98d-7a9357ca345e admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:06.644906 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077da5ec-8318-401c-b98d-7a9357ca345e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:06.646545 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077da5ec-8318-401c-b98d-7a9357ca345e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:06.646911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea74844-190f-4e42-bd5c-dfebf9e1743b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:06.652878 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8ee27-fe4b-4f81-8576-479deff51da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:06.653528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8ee27-fe4b-4f81-8576-479deff51da4 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:06.653799 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8ee27-fe4b-4f81-8576-479deff51da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:06.654073 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8ee27-fe4b-4f81-8576-479deff51da4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:06.654773 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8ee27-fe4b-4f81-8576-479deff51da4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:06.655080 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8ee27-fe4b-4f81-8576-479deff51da4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:06.656731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8ee27-fe4b-4f81-8576-479deff51da4 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:06.660066 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 950/1885] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:06.660943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077da5ec-8318-401c-b98d-7a9357ca345e admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:06.677075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077da5ec-8318-401c-b98d-7a9357ca345e admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:06.678498 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 936/1886] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:15:06.679560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8ee27-fe4b-4f81-8576-479deff51da4 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:06.682894 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f675b-f09b-4bda-b548-9794103998bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:06.683727 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f675b-f09b-4bda-b548-9794103998bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:06.684407 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f675b-f09b-4bda-b548-9794103998bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:06.684932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f675b-f09b-4bda-b548-9794103998bb admin admin] PATH_INFO: `/v3/projects/44b8ef0581aa49d38f3ad7e6d3d3e86d/users/0df6253b6d9243f6a255fd364f2da422/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:06.691885 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8ee27-fe4b-4f81-8576-479deff51da4 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:06.697314 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00eb17-7edf-489f-b587-6caab52b136c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:06.697950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00eb17-7edf-489f-b587-6caab52b136c admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:06.698220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00eb17-7edf-489f-b587-6caab52b136c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:06.698497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00eb17-7edf-489f-b587-6caab52b136c admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:06.699200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00eb17-7edf-489f-b587-6caab52b136c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:06.701062 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00eb17-7edf-489f-b587-6caab52b136c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:06.707447 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f675b-f09b-4bda-b548-9794103998bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b8ef0581aa49d38f3ad7e6d3d3e86d, user_id=0df6253b6d9243f6a255fd364f2da422, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:06.711331 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f675b-f09b-4bda-b548-9794103998bb admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:06.715076 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 937/1887] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:06.716440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00eb17-7edf-489f-b587-6caab52b136c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:06.733019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00eb17-7edf-489f-b587-6caab52b136c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:06.740458 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 951/1888] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:06 2022] PUT /identity/v3/projects/44b8ef0581aa49d38f3ad7e6d3d3e86d/users/0df6253b6d9243f6a255fd364f2da422/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:06.742812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f675b-f09b-4bda-b548-9794103998bb admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:06.755568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:06.835143 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:06.846273 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd2754-6ead-4d12-88ff-84d022cdb4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:06.847239 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd2754-6ead-4d12-88ff-84d022cdb4f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:06.847705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd2754-6ead-4d12-88ff-84d022cdb4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:06.848430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd2754-6ead-4d12-88ff-84d022cdb4f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:06.849560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd2754-6ead-4d12-88ff-84d022cdb4f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:06.853740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd2754-6ead-4d12-88ff-84d022cdb4f9 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:06.854400 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f675b-f09b-4bda-b548-9794103998bb admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:06.863220 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc74d7-7f15-4a75-a7f3-e1f8639bd05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:06.864361 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc74d7-7f15-4a75-a7f3-e1f8639bd05f admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:06.864869 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc74d7-7f15-4a75-a7f3-e1f8639bd05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:06.865322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc74d7-7f15-4a75-a7f3-e1f8639bd05f admin admin] PATH_INFO: `/v3/projects/1271e568c39841bdba343e6ac04bab20/users/0eaa3705e7a0489a9c0b94fbf69d08eb/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:06.891317 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc74d7-7f15-4a75-a7f3-e1f8639bd05f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1271e568c39841bdba343e6ac04bab20, user_id=0eaa3705e7a0489a9c0b94fbf69d08eb, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:06.895541 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc74d7-7f15-4a75-a7f3-e1f8639bd05f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:06.899609 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 938/1889] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:06 2022] POST /identity/v3/projects => generated 373 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:06.901072 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd2754-6ead-4d12-88ff-84d022cdb4f9 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:06.914977 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 952/1890] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:06 2022] PUT /identity/v3/projects/1271e568c39841bdba343e6ac04bab20/users/0eaa3705e7a0489a9c0b94fbf69d08eb/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:06.915834 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc74d7-7f15-4a75-a7f3-e1f8639bd05f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:07.003361 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd2754-6ead-4d12-88ff-84d022cdb4f9 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:07.022263 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc74d7-7f15-4a75-a7f3-e1f8639bd05f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:07.030880 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52b00a-1f62-4511-bc96-9b036fb77cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:07.031949 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52b00a-1f62-4511-bc96-9b036fb77cea admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:07.032447 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52b00a-1f62-4511-bc96-9b036fb77cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:07.033439 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52b00a-1f62-4511-bc96-9b036fb77cea admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:07.034870 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd52b00a-1f62-4511-bc96-9b036fb77cea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:07.038841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd52b00a-1f62-4511-bc96-9b036fb77cea admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:07.073843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 953/1891] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:06 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:07.075203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47004af7-591b-44c6-86b1-00fcd6439d1d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:07.075491 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47004af7-591b-44c6-86b1-00fcd6439d1d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:07.075768 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47004af7-591b-44c6-86b1-00fcd6439d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:07.088589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378ccaa-b6df-4928-b700-66f93ae214b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:07.089387 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378ccaa-b6df-4928-b700-66f93ae214b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:07.089835 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378ccaa-b6df-4928-b700-66f93ae214b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:07.090242 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378ccaa-b6df-4928-b700-66f93ae214b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:07.091172 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378ccaa-b6df-4928-b700-66f93ae214b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:07.093893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378ccaa-b6df-4928-b700-66f93ae214b7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:07.108012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 939/1892] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:07.109214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378ccaa-b6df-4928-b700-66f93ae214b7 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:07.115352 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-47004af7-591b-44c6-86b1-00fcd6439d1d None None] MFA Rules not processed for user `0eaa3705e7a0489a9c0b94fbf69d08eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:07.116866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-47004af7-591b-44c6-86b1-00fcd6439d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:07.124567 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378ccaa-b6df-4928-b700-66f93ae214b7 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:07.132417 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80129cbb-4bef-48c5-b5c0-a895818e87da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:07.133429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80129cbb-4bef-48c5-b5c0-a895818e87da admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:07.133777 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80129cbb-4bef-48c5-b5c0-a895818e87da admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:07.134124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80129cbb-4bef-48c5-b5c0-a895818e87da admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:07.135015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80129cbb-4bef-48c5-b5c0-a895818e87da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:07.136931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80129cbb-4bef-48c5-b5c0-a895818e87da admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:07.158043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 940/1893] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:07.161002 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80129cbb-4bef-48c5-b5c0-a895818e87da admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:07.178973 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80129cbb-4bef-48c5-b5c0-a895818e87da admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:07.187376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64b0d5-770e-4e6c-b6ab-109815b1d8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:07.188808 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64b0d5-770e-4e6c-b6ab-109815b1d8f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:07.190020 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64b0d5-770e-4e6c-b6ab-109815b1d8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:07.191019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64b0d5-770e-4e6c-b6ab-109815b1d8f4 admin admin] PATH_INFO: `/v3/projects/44b8ef0581aa49d38f3ad7e6d3d3e86d/users/0df6253b6d9243f6a255fd364f2da422/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:07.194404 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64b0d5-770e-4e6c-b6ab-109815b1d8f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b8ef0581aa49d38f3ad7e6d3d3e86d, user_id=0df6253b6d9243f6a255fd364f2da422, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:07.199517 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64b0d5-770e-4e6c-b6ab-109815b1d8f4 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:07.234702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 941/1894] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:07 2022] PUT /identity/v3/projects/44b8ef0581aa49d38f3ad7e6d3d3e86d/users/0df6253b6d9243f6a255fd364f2da422/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:07.234702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64b0d5-770e-4e6c-b6ab-109815b1d8f4 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:07.268680 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 954/1895] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:07.277052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753eefb5-28b7-46b5-a25e-98d083a775b2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:07.278017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753eefb5-28b7-46b5-a25e-98d083a775b2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:07.278017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753eefb5-28b7-46b5-a25e-98d083a775b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:07.335387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:07.426700 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-753eefb5-28b7-46b5-a25e-98d083a775b2 None None] MFA Rules not processed for user `0df6253b6d9243f6a255fd364f2da422`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:07.426700 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-753eefb5-28b7-46b5-a25e-98d083a775b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:07.446349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64b0d5-770e-4e6c-b6ab-109815b1d8f4 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:07.457194 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fce44c-a009-4ffc-a7c2-2a72a59a81cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:07.457194 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fce44c-a009-4ffc-a7c2-2a72a59a81cf admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:07.457684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fce44c-a009-4ffc-a7c2-2a72a59a81cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:07.458203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fce44c-a009-4ffc-a7c2-2a72a59a81cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:07.459506 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fce44c-a009-4ffc-a7c2-2a72a59a81cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:07.463116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fce44c-a009-4ffc-a7c2-2a72a59a81cf admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:07.469290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:07.502753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 942/1896] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:07.506504 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fce44c-a009-4ffc-a7c2-2a72a59a81cf admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:07.532594 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fce44c-a009-4ffc-a7c2-2a72a59a81cf admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:07.544496 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117604e-5a07-476f-99be-77416404b7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:07.546294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117604e-5a07-476f-99be-77416404b7e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:07.547268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117604e-5a07-476f-99be-77416404b7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:07.547783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117604e-5a07-476f-99be-77416404b7e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:07.549176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117604e-5a07-476f-99be-77416404b7e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:07.553016 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117604e-5a07-476f-99be-77416404b7e4 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:07.583552 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 955/1897] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:07 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:07.589558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753eefb5-28b7-46b5-a25e-98d083a775b2 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:07.630849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 943/1898] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:07 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:07.630849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117604e-5a07-476f-99be-77416404b7e4 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:07.630849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c117604e-5a07-476f-99be-77416404b7e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:07.636706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753eefb5-28b7-46b5-a25e-98d083a775b2 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:07.755079 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1725b3-eea4-49f3-9fec-ff9d29f4f006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:07.756179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1725b3-eea4-49f3-9fec-ff9d29f4f006 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:07.756639 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1725b3-eea4-49f3-9fec-ff9d29f4f006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:07.757095 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1725b3-eea4-49f3-9fec-ff9d29f4f006 admin admin] PATH_INFO: `/v3/projects/f124871e909842f4b74ef9fb483f5dd9/users/b24f1542cddf4fc9aa8003952e26cb96/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:07.769689 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117604e-5a07-476f-99be-77416404b7e4 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:07.777296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df7e4f-8bb6-41ea-a94c-7b2bde7e27c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:07.778397 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df7e4f-8bb6-41ea-a94c-7b2bde7e27c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:07.779828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df7e4f-8bb6-41ea-a94c-7b2bde7e27c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:07.780354 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df7e4f-8bb6-41ea-a94c-7b2bde7e27c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:07.781207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1725b3-eea4-49f3-9fec-ff9d29f4f006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f124871e909842f4b74ef9fb483f5dd9, user_id=b24f1542cddf4fc9aa8003952e26cb96, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:07.787496 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1725b3-eea4-49f3-9fec-ff9d29f4f006 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:07.820762 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 956/1899] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:07 2022] PUT /identity/v3/projects/f124871e909842f4b74ef9fb483f5dd9/users/b24f1542cddf4fc9aa8003952e26cb96/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:07.820762 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1725b3-eea4-49f3-9fec-ff9d29f4f006 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:07.896371 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df7e4f-8bb6-41ea-a94c-7b2bde7e27c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:07.898095 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df7e4f-8bb6-41ea-a94c-7b2bde7e27c3 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:07.914651 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 944/1900] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:07 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:07.916108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4aa722-c7f4-4fd6-8591-4236532be7fc None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:07.916407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4aa722-c7f4-4fd6-8591-4236532be7fc None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:07.916673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4aa722-c7f4-4fd6-8591-4236532be7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:07.932409 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1725b3-eea4-49f3-9fec-ff9d29f4f006 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:07.938215 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27883f6f-1a82-4475-ac63-ffaed7d8b7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:07.938993 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27883f6f-1a82-4475-ac63-ffaed7d8b7c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:07.939446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27883f6f-1a82-4475-ac63-ffaed7d8b7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:07.940424 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27883f6f-1a82-4475-ac63-ffaed7d8b7c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:07.940570 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27883f6f-1a82-4475-ac63-ffaed7d8b7c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:07.944447 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27883f6f-1a82-4475-ac63-ffaed7d8b7c5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:07.944666 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1c4aa722-c7f4-4fd6-8591-4236532be7fc None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:07.946784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1c4aa722-c7f4-4fd6-8591-4236532be7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:07.961617 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 945/1901] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:07.971957 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4aa722-c7f4-4fd6-8591-4236532be7fc None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:07.973156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 957/1902] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:07 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:07.974427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1c4aa722-c7f4-4fd6-8591-4236532be7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:07.979311 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27883f6f-1a82-4475-ac63-ffaed7d8b7c5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:07.985075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:07.995953 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27883f6f-1a82-4475-ac63-ffaed7d8b7c5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:08.002541 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335357cc-958d-4cd0-b943-352064c2c197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:08.003259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335357cc-958d-4cd0-b943-352064c2c197 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:08.003668 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335357cc-958d-4cd0-b943-352064c2c197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:08.003843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335357cc-958d-4cd0-b943-352064c2c197 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:08.004836 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335357cc-958d-4cd0-b943-352064c2c197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:08.007159 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335357cc-958d-4cd0-b943-352064c2c197 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:08.025638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 958/1903] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:15:08.027698 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335357cc-958d-4cd0-b943-352064c2c197 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:08.040540 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335357cc-958d-4cd0-b943-352064c2c197 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:08.055761 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e6508-8d8b-4b47-947d-ff7c971c8e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:08.056840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e6508-8d8b-4b47-947d-ff7c971c8e0e admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:08.057577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e6508-8d8b-4b47-947d-ff7c971c8e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:08.058041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e6508-8d8b-4b47-947d-ff7c971c8e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:08.059221 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e6508-8d8b-4b47-947d-ff7c971c8e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:08.063751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e6508-8d8b-4b47-947d-ff7c971c8e0e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:08.073919 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4aa722-c7f4-4fd6-8591-4236532be7fc None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:08.081878 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc503707-fbc7-4c6f-9695-4b010ecee52e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:08.092968 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc503707-fbc7-4c6f-9695-4b010ecee52e service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:08.092968 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc503707-fbc7-4c6f-9695-4b010ecee52e service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:08.092968 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc503707-fbc7-4c6f-9695-4b010ecee52e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:08.093522 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 959/1904] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:15:08.093522 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e6508-8d8b-4b47-947d-ff7c971c8e0e admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:08.104876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e6508-8d8b-4b47-947d-ff7c971c8e0e admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:08.105123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc503707-fbc7-4c6f-9695-4b010ecee52e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:08.106932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc503707-fbc7-4c6f-9695-4b010ecee52e service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:08.110874 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f77e1f-4780-43b4-b9eb-689f2980fe0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:08.111976 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f77e1f-4780-43b4-b9eb-689f2980fe0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:08.112281 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f77e1f-4780-43b4-b9eb-689f2980fe0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:08.112557 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f77e1f-4780-43b4-b9eb-689f2980fe0f admin admin] PATH_INFO: `/v3/projects/d9ff96d9dec5461f900b8fd73d4f5462/users/c17ccfaf8cc04caa9b9242123dbb958b/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:08.118783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 946/1905] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:15:08.119753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc503707-fbc7-4c6f-9695-4b010ecee52e service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:08.135327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f77e1f-4780-43b4-b9eb-689f2980fe0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ff96d9dec5461f900b8fd73d4f5462, user_id=c17ccfaf8cc04caa9b9242123dbb958b, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:08.138103 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f77e1f-4780-43b4-b9eb-689f2980fe0f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:08.157194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 960/1906] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:08 2022] PUT /identity/v3/projects/d9ff96d9dec5461f900b8fd73d4f5462/users/c17ccfaf8cc04caa9b9242123dbb958b/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:08.160556 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:08.169142 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaef090-946f-4624-bf59-30e800f15668 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:08.169792 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaef090-946f-4624-bf59-30e800f15668 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:08.170333 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaef090-946f-4624-bf59-30e800f15668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:08.225057 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-9aaef090-946f-4624-bf59-30e800f15668 None None] MFA Rules not processed for user `c17ccfaf8cc04caa9b9242123dbb958b`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:08.226460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9aaef090-946f-4624-bf59-30e800f15668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:08.238612 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc503707-fbc7-4c6f-9695-4b010ecee52e service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:08.343279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49efc28-bfaf-47f7-b166-2a772b64ce84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:08.343927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49efc28-bfaf-47f7-b166-2a772b64ce84 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:08.344265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49efc28-bfaf-47f7-b166-2a772b64ce84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:08.344599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49efc28-bfaf-47f7-b166-2a772b64ce84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:08.351399 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a49efc28-bfaf-47f7-b166-2a772b64ce84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:08.356465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 961/1907] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:15:08.365596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaef090-946f-4624-bf59-30e800f15668 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:08.528674 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49efc28-bfaf-47f7-b166-2a772b64ce84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:08.531575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49efc28-bfaf-47f7-b166-2a772b64ce84 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:08.554196 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 947/1908] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:08.556576 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaef090-946f-4624-bf59-30e800f15668 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:08.579931 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295873d4-6163-4dac-a75d-16a3d181da6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:08.581064 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295873d4-6163-4dac-a75d-16a3d181da6a admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:08.581450 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295873d4-6163-4dac-a75d-16a3d181da6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:08.582364 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295873d4-6163-4dac-a75d-16a3d181da6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:08.583053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295873d4-6163-4dac-a75d-16a3d181da6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:08.588367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295873d4-6163-4dac-a75d-16a3d181da6a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:08.738472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 962/1909] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:08 2022] POST /identity/v3/projects => generated 387 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:08.742771 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49efc28-bfaf-47f7-b166-2a772b64ce84 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:08.797423 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49efc28-bfaf-47f7-b166-2a772b64ce84 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:09.066970 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f0ec5-1261-4749-853b-93d78b490f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:09.067963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f0ec5-1261-4749-853b-93d78b490f49 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:09.068470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f0ec5-1261-4749-853b-93d78b490f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:09.069082 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f0ec5-1261-4749-853b-93d78b490f49 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:09.070274 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f0ec5-1261-4749-853b-93d78b490f49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:09.074780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f0ec5-1261-4749-853b-93d78b490f49 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:09.128601 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 948/1910] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:08 2022] POST /identity/v3/users => generated 300 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:15:09.139171 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295873d4-6163-4dac-a75d-16a3d181da6a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:09.166199 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295873d4-6163-4dac-a75d-16a3d181da6a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:09.188175 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f6490-757f-4b40-9aab-f99668176cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:09.189792 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f6490-757f-4b40-9aab-f99668176cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:09.190726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f6490-757f-4b40-9aab-f99668176cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:09.191230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f6490-757f-4b40-9aab-f99668176cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:09.192561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f6490-757f-4b40-9aab-f99668176cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:09.195831 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f6490-757f-4b40-9aab-f99668176cb3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:09.220163 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 963/1911] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:09.230880 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f0ec5-1261-4749-853b-93d78b490f49 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:09.261239 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f0ec5-1261-4749-853b-93d78b490f49 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:09.279498 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf63c6-2925-4d26-9979-3867fffc2048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:09.281181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf63c6-2925-4d26-9979-3867fffc2048 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:09.281562 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf63c6-2925-4d26-9979-3867fffc2048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:09.281899 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf63c6-2925-4d26-9979-3867fffc2048 admin admin] PATH_INFO: `/v3/projects/ac9adc7708ab4dd0a967ca465bab86ed/users/88205bbb257a4b34939629a35fbfbd64/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:09.319835 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf63c6-2925-4d26-9979-3867fffc2048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac9adc7708ab4dd0a967ca465bab86ed, user_id=88205bbb257a4b34939629a35fbfbd64, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:09.325587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf63c6-2925-4d26-9979-3867fffc2048 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:09.353113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:09.364980 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 949/1912] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:09 2022] PUT /identity/v3/projects/ac9adc7708ab4dd0a967ca465bab86ed/users/88205bbb257a4b34939629a35fbfbd64/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:09.370959 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f6490-757f-4b40-9aab-f99668176cb3 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:09.402172 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f6490-757f-4b40-9aab-f99668176cb3 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:09.439987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:09.513024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd259cae-0e7c-4c41-967b-835cc1f0d324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:09.513860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd259cae-0e7c-4c41-967b-835cc1f0d324 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:09.514273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd259cae-0e7c-4c41-967b-835cc1f0d324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:09.514712 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd259cae-0e7c-4c41-967b-835cc1f0d324 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:09.515625 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd259cae-0e7c-4c41-967b-835cc1f0d324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:09.518331 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd259cae-0e7c-4c41-967b-835cc1f0d324 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:09.541550 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 964/1913] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:09.548615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf63c6-2925-4d26-9979-3867fffc2048 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:09.561589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf63c6-2925-4d26-9979-3867fffc2048 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:09.567737 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701db03d-d80a-41ca-9c45-e178964b508e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:09.568602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701db03d-d80a-41ca-9c45-e178964b508e admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:09.568923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701db03d-d80a-41ca-9c45-e178964b508e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:09.569204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701db03d-d80a-41ca-9c45-e178964b508e admin admin] PATH_INFO: `/v3/projects/ac9adc7708ab4dd0a967ca465bab86ed/users/88205bbb257a4b34939629a35fbfbd64/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:09.571676 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701db03d-d80a-41ca-9c45-e178964b508e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac9adc7708ab4dd0a967ca465bab86ed, user_id=88205bbb257a4b34939629a35fbfbd64, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:09.574548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701db03d-d80a-41ca-9c45-e178964b508e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:09.590642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 950/1914] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:09 2022] PUT /identity/v3/projects/ac9adc7708ab4dd0a967ca465bab86ed/users/88205bbb257a4b34939629a35fbfbd64/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:09.615447 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c687e-3c22-48b2-b5d6-42cfe9a0ccea None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:09.615963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c687e-3c22-48b2-b5d6-42cfe9a0ccea None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:09.616472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c687e-3c22-48b2-b5d6-42cfe9a0ccea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:09.645538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a7a2a-e953-4099-8245-860df90df75a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:09.645859 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a7a2a-e953-4099-8245-860df90df75a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:09.646095 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a7a2a-e953-4099-8245-860df90df75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:09.676815 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-492c687e-3c22-48b2-b5d6-42cfe9a0ccea None None] MFA Rules not processed for user `88205bbb257a4b34939629a35fbfbd64`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:09.679507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-492c687e-3c22-48b2-b5d6-42cfe9a0ccea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:09.683311 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-158a7a2a-e953-4099-8245-860df90df75a None None] MFA Rules not processed for user `b24f1542cddf4fc9aa8003952e26cb96`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:09.684554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-158a7a2a-e953-4099-8245-860df90df75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:09.826154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 965/1915] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:09 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:09.826412 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 951/1916] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:09.837009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c687e-3c22-48b2-b5d6-42cfe9a0ccea None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:09.841164 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a7a2a-e953-4099-8245-860df90df75a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:09.843751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-158a7a2a-e953-4099-8245-860df90df75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:09.953926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c687e-3c22-48b2-b5d6-42cfe9a0ccea None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:09.962620 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9a2a3-9bc1-46b7-b9f4-0bac47f28e38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:09.963761 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9a2a3-9bc1-46b7-b9f4-0bac47f28e38 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:09.964282 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9a2a3-9bc1-46b7-b9f4-0bac47f28e38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:09.964769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9a2a3-9bc1-46b7-b9f4-0bac47f28e38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:09.972231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a7a2a-e953-4099-8245-860df90df75a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:09.979929 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0dfe3-18ca-433f-9061-989a974231cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:09.980269 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0dfe3-18ca-433f-9061-989a974231cb service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:09.980450 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0dfe3-18ca-433f-9061-989a974231cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:09.980884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0dfe3-18ca-433f-9061-989a974231cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:09.983583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9a2a3-9bc1-46b7-b9f4-0bac47f28e38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:09.987416 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9a2a3-9bc1-46b7-b9f4-0bac47f28e38 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:09.997796 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0dfe3-18ca-433f-9061-989a974231cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:10.000901 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0dfe3-18ca-433f-9061-989a974231cb service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:10.005322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 966/1917] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:10.018326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 952/1918] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:11.349325 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9a2a3-9bc1-46b7-b9f4-0bac47f28e38 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:11.360284 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9a2a3-9bc1-46b7-b9f4-0bac47f28e38 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:11.366240 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ee34b-a4f6-40af-99a9-6099133788d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:11.367207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ee34b-a4f6-40af-99a9-6099133788d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:11.367632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ee34b-a4f6-40af-99a9-6099133788d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:11.368084 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ee34b-a4f6-40af-99a9-6099133788d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:11.435823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:11.458631 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ee34b-a4f6-40af-99a9-6099133788d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:11.460230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ee34b-a4f6-40af-99a9-6099133788d7 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:11.472578 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 967/1919] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:13.981247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f210669-d5be-4b1a-9653-e9b0d3c0492b None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:13.981844 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f210669-d5be-4b1a-9653-e9b0d3c0492b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:13.982268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f210669-d5be-4b1a-9653-e9b0d3c0492b None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:13.984679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 953/1920] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 6 16:15:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:15:14.003282 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638c059-35be-426f-84e6-486d62a884a3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:14.003838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638c059-35be-426f-84e6-486d62a884a3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:14.004325 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638c059-35be-426f-84e6-486d62a884a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:14.038248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-5638c059-35be-426f-84e6-486d62a884a3 None None] MFA Rules not processed for user `6afd4f6ffdae4b16aa5ba757e68c18c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:14.040955 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5638c059-35be-426f-84e6-486d62a884a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:14.186899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 968/1921] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 6 16:15:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:15:14.212767 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f210669-d5be-4b1a-9653-e9b0d3c0492b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:14.216865 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5f210669-d5be-4b1a-9653-e9b0d3c0492b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:14.260903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f210669-d5be-4b1a-9653-e9b0d3c0492b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:14.277513 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7daf79b-6167-4a76-b5cf-288e1af9d325 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:14.278970 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daf79b-6167-4a76-b5cf-288e1af9d325 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:14.279393 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daf79b-6167-4a76-b5cf-288e1af9d325 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:14.279844 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daf79b-6167-4a76-b5cf-288e1af9d325 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:14.304102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7daf79b-6167-4a76-b5cf-288e1af9d325 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:14.307281 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7daf79b-6167-4a76-b5cf-288e1af9d325 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:14.331361 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 954/1922] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:16.439002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ee764-9616-464d-a100-6c4fbcd3c001 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:16.440387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ee764-9616-464d-a100-6c4fbcd3c001 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:16.440387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ee764-9616-464d-a100-6c4fbcd3c001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:16.524297 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-800ee764-9616-464d-a100-6c4fbcd3c001 None None] MFA Rules not processed for user `b24f1542cddf4fc9aa8003952e26cb96`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:16.524297 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-800ee764-9616-464d-a100-6c4fbcd3c001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:16.547976 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 969/1923] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:19.055325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7daf79b-6167-4a76-b5cf-288e1af9d325 service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:19.057178 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d7daf79b-6167-4a76-b5cf-288e1af9d325 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:19.148058 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7daf79b-6167-4a76-b5cf-288e1af9d325 service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:19.155197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcd63b-da16-4e30-ae18-050c3c1a9f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:19.155979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcd63b-da16-4e30-ae18-050c3c1a9f7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:19.156378 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcd63b-da16-4e30-ae18-050c3c1a9f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:19.156769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcd63b-da16-4e30-ae18-050c3c1a9f7b admin admin] PATH_INFO: `/v3/users/59adfeaccb994120ae348fb7429e2d63` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:19.159330 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcd63b-da16-4e30-ae18-050c3c1a9f7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=59adfeaccb994120ae348fb7429e2d63)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:19.161598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcd63b-da16-4e30-ae18-050c3c1a9f7b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:19.168160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:19.242176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-83dcd63b-da16-4e30-ae18-050c3c1a9f7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59adfeaccb994120ae348fb7429e2d63'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:19.274116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-83dcd63b-da16-4e30-ae18-050c3c1a9f7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59adfeaccb994120ae348fb7429e2d63'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:19.296475 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-83dcd63b-da16-4e30-ae18-050c3c1a9f7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59adfeaccb994120ae348fb7429e2d63'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:19.297386 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-83dcd63b-da16-4e30-ae18-050c3c1a9f7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59adfeaccb994120ae348fb7429e2d63'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:19.299010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-83dcd63b-da16-4e30-ae18-050c3c1a9f7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59adfeaccb994120ae348fb7429e2d63'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:19.322746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 955/1924] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:19 2022] DELETE /identity/v3/users/59adfeaccb994120ae348fb7429e2d63 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:19.328288 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d51be-a5b9-463d-a261-11d3d90f1f32 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:19.329119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d51be-a5b9-463d-a261-11d3d90f1f32 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:19.329256 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d51be-a5b9-463d-a261-11d3d90f1f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:19.357600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ca8d51be-a5b9-463d-a261-11d3d90f1f32 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:19.359685 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ca8d51be-a5b9-463d-a261-11d3d90f1f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:19.485504 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 970/1925] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:15:19.502550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcd63b-da16-4e30-ae18-050c3c1a9f7b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:19.505292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-83dcd63b-da16-4e30-ae18-050c3c1a9f7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:19.710821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcd63b-da16-4e30-ae18-050c3c1a9f7b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:19.717485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a44d9-c00f-4a3e-b218-18bc6bae1cc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:19.718858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a44d9-c00f-4a3e-b218-18bc6bae1cc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:19.719795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a44d9-c00f-4a3e-b218-18bc6bae1cc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:19.720501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a44d9-c00f-4a3e-b218-18bc6bae1cc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:19.739348 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a44d9-c00f-4a3e-b218-18bc6bae1cc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:19.743385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a44d9-c00f-4a3e-b218-18bc6bae1cc4 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:19.765433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 956/1926] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:20.192187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d51be-a5b9-463d-a261-11d3d90f1f32 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:20.193645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ca8d51be-a5b9-463d-a261-11d3d90f1f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:20.430515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d51be-a5b9-463d-a261-11d3d90f1f32 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:20.456816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee67c68-3bba-419f-bbf2-cf972f309cb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:20.458378 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee67c68-3bba-419f-bbf2-cf972f309cb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:20.459103 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee67c68-3bba-419f-bbf2-cf972f309cb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:20.459793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee67c68-3bba-419f-bbf2-cf972f309cb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:20.463389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9ee67c68-3bba-419f-bbf2-cf972f309cb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:20.585710 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:20.872833 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee67c68-3bba-419f-bbf2-cf972f309cb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:20.876132 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee67c68-3bba-419f-bbf2-cf972f309cb6 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:20.918135 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 971/1927] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 726 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:15:21.373911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a44d9-c00f-4a3e-b218-18bc6bae1cc4 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:21.376296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0a2a44d9-c00f-4a3e-b218-18bc6bae1cc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:21.430650 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a44d9-c00f-4a3e-b218-18bc6bae1cc4 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:21.459180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00878c90-536b-4690-98b3-5df18234dec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:21.462083 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00878c90-536b-4690-98b3-5df18234dec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:21.462631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00878c90-536b-4690-98b3-5df18234dec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:21.463152 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00878c90-536b-4690-98b3-5df18234dec9 admin admin] PATH_INFO: `/v3/projects/795ca90477604c72912ba07154071959` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:21.470353 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00878c90-536b-4690-98b3-5df18234dec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=795ca90477604c72912ba07154071959)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:21.509538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00878c90-536b-4690-98b3-5df18234dec9 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:21.748055 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-00878c90-536b-4690-98b3-5df18234dec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '795ca90477604c72912ba07154071959', 'request_id': 'req-00878c90-536b-4690-98b3-5df18234dec9'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:21.799922 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-00878c90-536b-4690-98b3-5df18234dec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '795ca90477604c72912ba07154071959', 'request_id': 'req-00878c90-536b-4690-98b3-5df18234dec9'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:21.826201 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-00878c90-536b-4690-98b3-5df18234dec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '795ca90477604c72912ba07154071959', 'request_id': 'req-00878c90-536b-4690-98b3-5df18234dec9'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:21.875289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-00878c90-536b-4690-98b3-5df18234dec9 admin admin] The token cache is being invalidate because project 795ca90477604c72912ba07154071959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:21.876114 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-00878c90-536b-4690-98b3-5df18234dec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:21.880091 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-00878c90-536b-4690-98b3-5df18234dec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:21.885262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 957/1928] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:21 2022] DELETE /identity/v3/projects/795ca90477604c72912ba07154071959 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:21.892262 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee67c68-3bba-419f-bbf2-cf972f309cb6 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:21.927258 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee67c68-3bba-419f-bbf2-cf972f309cb6 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:22.073815 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a94508-6daa-4254-9cd7-8a2371fcd218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:22.075643 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a94508-6daa-4254-9cd7-8a2371fcd218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:22.076333 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a94508-6daa-4254-9cd7-8a2371fcd218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:22.077242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a94508-6daa-4254-9cd7-8a2371fcd218 admin admin] PATH_INFO: `/v3/users/3732a18629f547efbb85dead7e10cc95` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:22.080096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a94508-6daa-4254-9cd7-8a2371fcd218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3732a18629f547efbb85dead7e10cc95)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:22.085642 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a94508-6daa-4254-9cd7-8a2371fcd218 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:22.192888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-81a94508-6daa-4254-9cd7-8a2371fcd218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3732a18629f547efbb85dead7e10cc95'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:22.205916 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-81a94508-6daa-4254-9cd7-8a2371fcd218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3732a18629f547efbb85dead7e10cc95'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:22.223294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-81a94508-6daa-4254-9cd7-8a2371fcd218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3732a18629f547efbb85dead7e10cc95'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:22.224088 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-81a94508-6daa-4254-9cd7-8a2371fcd218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3732a18629f547efbb85dead7e10cc95'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:22.234918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-81a94508-6daa-4254-9cd7-8a2371fcd218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3732a18629f547efbb85dead7e10cc95'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:22.247083 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 972/1929] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:21 2022] DELETE /identity/v3/users/3732a18629f547efbb85dead7e10cc95 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:22.888403 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00878c90-536b-4690-98b3-5df18234dec9 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:22.890502 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-00878c90-536b-4690-98b3-5df18234dec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:22.914296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00878c90-536b-4690-98b3-5df18234dec9 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:22.921860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:23.027319 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3673003-ed19-4d64-8431-b8d14e1d8598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:23.027972 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3673003-ed19-4d64-8431-b8d14e1d8598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:23.028249 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3673003-ed19-4d64-8431-b8d14e1d8598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:23.028520 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3673003-ed19-4d64-8431-b8d14e1d8598 admin admin] PATH_INFO: `/v3/projects/1adc57dadc6743e5ab8f49274878ff2a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:23.029868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3673003-ed19-4d64-8431-b8d14e1d8598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1adc57dadc6743e5ab8f49274878ff2a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:23.031898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3673003-ed19-4d64-8431-b8d14e1d8598 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:23.098424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d3673003-ed19-4d64-8431-b8d14e1d8598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1adc57dadc6743e5ab8f49274878ff2a', 'request_id': 'req-d3673003-ed19-4d64-8431-b8d14e1d8598'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:23.109415 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d3673003-ed19-4d64-8431-b8d14e1d8598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1adc57dadc6743e5ab8f49274878ff2a', 'request_id': 'req-d3673003-ed19-4d64-8431-b8d14e1d8598'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:23.118903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d3673003-ed19-4d64-8431-b8d14e1d8598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1adc57dadc6743e5ab8f49274878ff2a', 'request_id': 'req-d3673003-ed19-4d64-8431-b8d14e1d8598'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:23.133245 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d3673003-ed19-4d64-8431-b8d14e1d8598 admin admin] The token cache is being invalidate because project 1adc57dadc6743e5ab8f49274878ff2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:23.133563 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d3673003-ed19-4d64-8431-b8d14e1d8598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:23.134401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d3673003-ed19-4d64-8431-b8d14e1d8598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:23.136151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 958/1930] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:22 2022] DELETE /identity/v3/projects/1adc57dadc6743e5ab8f49274878ff2a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:23.143095 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9478d-890b-40c9-94d4-dc685c4fab98 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:23.143971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9478d-890b-40c9-94d4-dc685c4fab98 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:23.144503 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9478d-890b-40c9-94d4-dc685c4fab98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:23.175286 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a4f9478d-890b-40c9-94d4-dc685c4fab98 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:23.176980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a4f9478d-890b-40c9-94d4-dc685c4fab98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:23.243439 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:23.281086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 973/1931] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:23.291359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f315002-d80c-40d1-93ea-c2f3efd62fe1 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:23.291784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f315002-d80c-40d1-93ea-c2f3efd62fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:23.292070 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f315002-d80c-40d1-93ea-c2f3efd62fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:23.317560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0f315002-d80c-40d1-93ea-c2f3efd62fe1 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:23.318860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0f315002-d80c-40d1-93ea-c2f3efd62fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:23.327255 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 959/1932] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:23.332323 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9478d-890b-40c9-94d4-dc685c4fab98 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:23.342426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9478d-890b-40c9-94d4-dc685c4fab98 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:23.348024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd78a8-d8c5-4f85-a796-6d64dad24264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:23.348770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd78a8-d8c5-4f85-a796-6d64dad24264 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:23.349041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd78a8-d8c5-4f85-a796-6d64dad24264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:23.349387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd78a8-d8c5-4f85-a796-6d64dad24264 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:23.349992 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd78a8-d8c5-4f85-a796-6d64dad24264 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:23.350352 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd78a8-d8c5-4f85-a796-6d64dad24264 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:23.352152 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd78a8-d8c5-4f85-a796-6d64dad24264 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:23.373022 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 974/1933] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:23.382791 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f315002-d80c-40d1-93ea-c2f3efd62fe1 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:23.395646 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f315002-d80c-40d1-93ea-c2f3efd62fe1 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:23.406862 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde06adb-fa53-4524-a146-d0c9ec8b30ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:23.407822 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde06adb-fa53-4524-a146-d0c9ec8b30ce admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:23.408146 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde06adb-fa53-4524-a146-d0c9ec8b30ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:23.408419 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde06adb-fa53-4524-a146-d0c9ec8b30ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:23.409396 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde06adb-fa53-4524-a146-d0c9ec8b30ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:23.411404 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde06adb-fa53-4524-a146-d0c9ec8b30ce admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:23.446436 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 960/1934] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:23 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:23.451021 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd78a8-d8c5-4f85-a796-6d64dad24264 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:23.462278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd78a8-d8c5-4f85-a796-6d64dad24264 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:23.526324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ba198-d3c7-4307-9d85-917c71e1ce19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:23.526840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ba198-d3c7-4307-9d85-917c71e1ce19 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:23.527120 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ba198-d3c7-4307-9d85-917c71e1ce19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:23.527396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ba198-d3c7-4307-9d85-917c71e1ce19 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:23.528056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ba198-d3c7-4307-9d85-917c71e1ce19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:23.530004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ba198-d3c7-4307-9d85-917c71e1ce19 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:23.555483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 975/1935] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:23 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:15:23.561783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde06adb-fa53-4524-a146-d0c9ec8b30ce admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:23.576052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde06adb-fa53-4524-a146-d0c9ec8b30ce admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:23.583181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb7cfe-73bc-4a73-a221-cc3a2f6d980e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:23.583792 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb7cfe-73bc-4a73-a221-cc3a2f6d980e admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:23.584069 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb7cfe-73bc-4a73-a221-cc3a2f6d980e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:23.584338 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb7cfe-73bc-4a73-a221-cc3a2f6d980e admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:23.584999 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb7cfe-73bc-4a73-a221-cc3a2f6d980e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:23.586648 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb7cfe-73bc-4a73-a221-cc3a2f6d980e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:23.599931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 961/1936] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:23.604065 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ba198-d3c7-4307-9d85-917c71e1ce19 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:23.616196 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ba198-d3c7-4307-9d85-917c71e1ce19 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:23.621023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b2645-8e74-471e-b908-204765c4104f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:23.621645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b2645-8e74-471e-b908-204765c4104f admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:23.621927 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b2645-8e74-471e-b908-204765c4104f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:23.622194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b2645-8e74-471e-b908-204765c4104f admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:23.622877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b2645-8e74-471e-b908-204765c4104f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:23.624493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b2645-8e74-471e-b908-204765c4104f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:23.636527 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 976/1937] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:23.640854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb7cfe-73bc-4a73-a221-cc3a2f6d980e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:23.651825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb7cfe-73bc-4a73-a221-cc3a2f6d980e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:23.656638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9010b14-09dd-4b3b-8ac2-5ae125f31df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:23.657309 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9010b14-09dd-4b3b-8ac2-5ae125f31df5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:23.657584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9010b14-09dd-4b3b-8ac2-5ae125f31df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:23.658533 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9010b14-09dd-4b3b-8ac2-5ae125f31df5 admin admin] PATH_INFO: `/v3/projects/08307494e0db4b889eab7d06a46b123f/users/e769c55bb2d84563865aced558c04afc/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:23.684206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9010b14-09dd-4b3b-8ac2-5ae125f31df5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08307494e0db4b889eab7d06a46b123f, user_id=e769c55bb2d84563865aced558c04afc, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:23.689181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9010b14-09dd-4b3b-8ac2-5ae125f31df5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:23.707274 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 962/1938] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:23 2022] PUT /identity/v3/projects/08307494e0db4b889eab7d06a46b123f/users/e769c55bb2d84563865aced558c04afc/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:23.717731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904432b5-a384-4a6e-a9a0-e26359751eb0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:23.718022 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904432b5-a384-4a6e-a9a0-e26359751eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:23.718304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904432b5-a384-4a6e-a9a0-e26359751eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:23.764096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-904432b5-a384-4a6e-a9a0-e26359751eb0 None None] MFA Rules not processed for user `e769c55bb2d84563865aced558c04afc`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:23.765532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-904432b5-a384-4a6e-a9a0-e26359751eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:23.870166 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 977/1939] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:23.875075 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9010b14-09dd-4b3b-8ac2-5ae125f31df5 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:23.926583 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:23.961598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9010b14-09dd-4b3b-8ac2-5ae125f31df5 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:23.971448 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f3f68-44d2-4d6e-9ff7-45a435cd1437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:23.971448 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f3f68-44d2-4d6e-9ff7-45a435cd1437 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:23.975385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f3f68-44d2-4d6e-9ff7-45a435cd1437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:23.976365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f3f68-44d2-4d6e-9ff7-45a435cd1437 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:23.977805 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f3f68-44d2-4d6e-9ff7-45a435cd1437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:23.982367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f3f68-44d2-4d6e-9ff7-45a435cd1437 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:24.042755 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 963/1940] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:23 2022] POST /identity/v3/projects => generated 387 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:24.044369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904432b5-a384-4a6e-a9a0-e26359751eb0 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:24.058425 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904432b5-a384-4a6e-a9a0-e26359751eb0 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:24.152919 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:24.186960 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccef4c-9add-4130-b44b-37e394b67d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:24.188709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccef4c-9add-4130-b44b-37e394b67d90 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:24.190200 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccef4c-9add-4130-b44b-37e394b67d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:24.190969 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccef4c-9add-4130-b44b-37e394b67d90 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:24.192829 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccef4c-9add-4130-b44b-37e394b67d90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:24.197271 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccef4c-9add-4130-b44b-37e394b67d90 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:24.257778 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 978/1941] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:24 2022] POST /identity/v3/users => generated 300 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:24.260094 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f3f68-44d2-4d6e-9ff7-45a435cd1437 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:24.285882 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f3f68-44d2-4d6e-9ff7-45a435cd1437 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:24.314625 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f790db-d7fe-4d60-9096-d75b95eb8cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:24.315927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f790db-d7fe-4d60-9096-d75b95eb8cbc admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:24.316643 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f790db-d7fe-4d60-9096-d75b95eb8cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:24.317296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f790db-d7fe-4d60-9096-d75b95eb8cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:24.318515 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f790db-d7fe-4d60-9096-d75b95eb8cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:24.324617 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f790db-d7fe-4d60-9096-d75b95eb8cbc admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:24.349002 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 964/1942] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:15:24.353558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccef4c-9add-4130-b44b-37e394b67d90 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:24.372327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccef4c-9add-4130-b44b-37e394b67d90 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:24.386372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726d9ed-fae4-4297-8b47-885315dbf0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:24.387181 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726d9ed-fae4-4297-8b47-885315dbf0ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:24.387605 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726d9ed-fae4-4297-8b47-885315dbf0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:24.387932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726d9ed-fae4-4297-8b47-885315dbf0ad admin admin] PATH_INFO: `/v3/projects/de6e36cee9234146abc628e21fdf05ea/users/249507288a304b67b80642f2ce7e465d/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:24.410905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726d9ed-fae4-4297-8b47-885315dbf0ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=de6e36cee9234146abc628e21fdf05ea, user_id=249507288a304b67b80642f2ce7e465d, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:24.414051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726d9ed-fae4-4297-8b47-885315dbf0ad admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:24.434344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 979/1943] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:24 2022] PUT /identity/v3/projects/de6e36cee9234146abc628e21fdf05ea/users/249507288a304b67b80642f2ce7e465d/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:24.439422 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f790db-d7fe-4d60-9096-d75b95eb8cbc admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:24.455218 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f790db-d7fe-4d60-9096-d75b95eb8cbc admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:24.545958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049519b6-b954-4ac4-813b-68ec222c7ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:24.546480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049519b6-b954-4ac4-813b-68ec222c7ddc admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:24.546914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049519b6-b954-4ac4-813b-68ec222c7ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:24.547190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049519b6-b954-4ac4-813b-68ec222c7ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:24.547894 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049519b6-b954-4ac4-813b-68ec222c7ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:24.549776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049519b6-b954-4ac4-813b-68ec222c7ddc admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:24.565050 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 965/1944] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:24.570445 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726d9ed-fae4-4297-8b47-885315dbf0ad admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:24.581688 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726d9ed-fae4-4297-8b47-885315dbf0ad admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:24.586877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee1e92-bead-450b-8a80-4a0778da461d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:24.587545 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee1e92-bead-450b-8a80-4a0778da461d admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:24.587820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee1e92-bead-450b-8a80-4a0778da461d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:24.588095 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee1e92-bead-450b-8a80-4a0778da461d admin admin] PATH_INFO: `/v3/projects/de6e36cee9234146abc628e21fdf05ea/users/249507288a304b67b80642f2ce7e465d/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:24.590075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee1e92-bead-450b-8a80-4a0778da461d admin admin] RBAC: Authorizing `identity:create_grant(project_id=de6e36cee9234146abc628e21fdf05ea, user_id=249507288a304b67b80642f2ce7e465d, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:24.592759 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee1e92-bead-450b-8a80-4a0778da461d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:24.605041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 980/1945] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:24 2022] PUT /identity/v3/projects/de6e36cee9234146abc628e21fdf05ea/users/249507288a304b67b80642f2ce7e465d/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:24.616106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39998d9d-2bf5-405d-81b1-d33458254b56 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:24.616441 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39998d9d-2bf5-405d-81b1-d33458254b56 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:24.616762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39998d9d-2bf5-405d-81b1-d33458254b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:24.651028 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-39998d9d-2bf5-405d-81b1-d33458254b56 None None] MFA Rules not processed for user `249507288a304b67b80642f2ce7e465d`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:24.652269 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-39998d9d-2bf5-405d-81b1-d33458254b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:24.739806 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 966/1946] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:24.751443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee1e92-bead-450b-8a80-4a0778da461d admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:24.753746 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-feee1e92-bead-450b-8a80-4a0778da461d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:24.817782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee1e92-bead-450b-8a80-4a0778da461d admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:24.821667 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c5a54-9757-445f-a4a6-54304418781a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:24.822302 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c5a54-9757-445f-a4a6-54304418781a service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:24.822580 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c5a54-9757-445f-a4a6-54304418781a service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:24.822909 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c5a54-9757-445f-a4a6-54304418781a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:24.834008 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c5a54-9757-445f-a4a6-54304418781a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:24.835496 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c5a54-9757-445f-a4a6-54304418781a service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:24.846308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 981/1947] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:15:25.818294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39998d9d-2bf5-405d-81b1-d33458254b56 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:25.858510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:25.895474 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39998d9d-2bf5-405d-81b1-d33458254b56 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:25.900561 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d26d8-5de7-4684-b8d1-ab4882c92b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:25.901219 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d26d8-5de7-4684-b8d1-ab4882c92b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:25.901502 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d26d8-5de7-4684-b8d1-ab4882c92b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:25.901777 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d26d8-5de7-4684-b8d1-ab4882c92b21 admin admin] PATH_INFO: `/v3/users/e769c55bb2d84563865aced558c04afc` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:25.903135 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d26d8-5de7-4684-b8d1-ab4882c92b21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e769c55bb2d84563865aced558c04afc)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:25.905116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d26d8-5de7-4684-b8d1-ab4882c92b21 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:25.964328 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7c0d26d8-5de7-4684-b8d1-ab4882c92b21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e769c55bb2d84563865aced558c04afc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:25.973265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7c0d26d8-5de7-4684-b8d1-ab4882c92b21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e769c55bb2d84563865aced558c04afc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:25.987684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7c0d26d8-5de7-4684-b8d1-ab4882c92b21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e769c55bb2d84563865aced558c04afc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:25.988452 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7c0d26d8-5de7-4684-b8d1-ab4882c92b21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e769c55bb2d84563865aced558c04afc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:25.989058 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7c0d26d8-5de7-4684-b8d1-ab4882c92b21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e769c55bb2d84563865aced558c04afc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:26.004303 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 967/1948] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:25 2022] DELETE /identity/v3/users/e769c55bb2d84563865aced558c04afc => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:26.008759 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c7fcd-5d8c-4099-a6df-b121963fdb00 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:26.009623 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c7fcd-5d8c-4099-a6df-b121963fdb00 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:26.009623 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c7fcd-5d8c-4099-a6df-b121963fdb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:26.031868 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a11c7fcd-5d8c-4099-a6df-b121963fdb00 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:26.033430 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a11c7fcd-5d8c-4099-a6df-b121963fdb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:26.115361 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 982/1949] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:26.129192 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d26d8-5de7-4684-b8d1-ab4882c92b21 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:26.131669 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7c0d26d8-5de7-4684-b8d1-ab4882c92b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:26.233057 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d26d8-5de7-4684-b8d1-ab4882c92b21 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:26.239997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495d802-c0ed-4e96-8df8-cbaaddb9e913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:26.240647 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495d802-c0ed-4e96-8df8-cbaaddb9e913 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:26.241134 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495d802-c0ed-4e96-8df8-cbaaddb9e913 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:26.241410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495d802-c0ed-4e96-8df8-cbaaddb9e913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:26.255338 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495d802-c0ed-4e96-8df8-cbaaddb9e913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:26.257052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495d802-c0ed-4e96-8df8-cbaaddb9e913 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:26.271516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 968/1950] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:26.635726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c7fcd-5d8c-4099-a6df-b121963fdb00 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:26.637546 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a11c7fcd-5d8c-4099-a6df-b121963fdb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:26.654283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c7fcd-5d8c-4099-a6df-b121963fdb00 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:26.659511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6d6fb-efb1-43c3-afd7-fcca554f7450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:26.660197 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6d6fb-efb1-43c3-afd7-fcca554f7450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:26.660453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6d6fb-efb1-43c3-afd7-fcca554f7450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:26.660727 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6d6fb-efb1-43c3-afd7-fcca554f7450 admin admin] PATH_INFO: `/v3/projects/08307494e0db4b889eab7d06a46b123f` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:26.661896 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6d6fb-efb1-43c3-afd7-fcca554f7450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08307494e0db4b889eab7d06a46b123f)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:26.663853 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6d6fb-efb1-43c3-afd7-fcca554f7450 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:26.713147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:26.734401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a8b6d6fb-efb1-43c3-afd7-fcca554f7450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08307494e0db4b889eab7d06a46b123f', 'request_id': 'req-a8b6d6fb-efb1-43c3-afd7-fcca554f7450'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:26.742046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a8b6d6fb-efb1-43c3-afd7-fcca554f7450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08307494e0db4b889eab7d06a46b123f', 'request_id': 'req-a8b6d6fb-efb1-43c3-afd7-fcca554f7450'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:26.748080 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a8b6d6fb-efb1-43c3-afd7-fcca554f7450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08307494e0db4b889eab7d06a46b123f', 'request_id': 'req-a8b6d6fb-efb1-43c3-afd7-fcca554f7450'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:26.764926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a8b6d6fb-efb1-43c3-afd7-fcca554f7450 admin admin] The token cache is being invalidate because project 08307494e0db4b889eab7d06a46b123f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:26.765279 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a8b6d6fb-efb1-43c3-afd7-fcca554f7450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:26.766111 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a8b6d6fb-efb1-43c3-afd7-fcca554f7450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:26.768002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 983/1951] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:26 2022] DELETE /identity/v3/projects/08307494e0db4b889eab7d06a46b123f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:26.772005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495d802-c0ed-4e96-8df8-cbaaddb9e913 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:26.783454 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495d802-c0ed-4e96-8df8-cbaaddb9e913 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:26.847764 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:26.861287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87ffd2-a8c6-4585-9da0-0fd1de0e8141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:26.862050 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87ffd2-a8c6-4585-9da0-0fd1de0e8141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:26.862338 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87ffd2-a8c6-4585-9da0-0fd1de0e8141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:26.862634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87ffd2-a8c6-4585-9da0-0fd1de0e8141 admin admin] PATH_INFO: `/v3/users/249507288a304b67b80642f2ce7e465d` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:26.864210 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a87ffd2-a8c6-4585-9da0-0fd1de0e8141 admin admin] RBAC: Authorizing `identity:delete_user(user_id=249507288a304b67b80642f2ce7e465d)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:26.866177 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a87ffd2-a8c6-4585-9da0-0fd1de0e8141 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:26.925441 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7a87ffd2-a8c6-4585-9da0-0fd1de0e8141 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '249507288a304b67b80642f2ce7e465d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:26.934364 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7a87ffd2-a8c6-4585-9da0-0fd1de0e8141 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '249507288a304b67b80642f2ce7e465d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:26.949649 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7a87ffd2-a8c6-4585-9da0-0fd1de0e8141 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '249507288a304b67b80642f2ce7e465d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:26.950489 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7a87ffd2-a8c6-4585-9da0-0fd1de0e8141 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '249507288a304b67b80642f2ce7e465d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:26.951119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7a87ffd2-a8c6-4585-9da0-0fd1de0e8141 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '249507288a304b67b80642f2ce7e465d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:26.972166 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 969/1952] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:26 2022] DELETE /identity/v3/users/249507288a304b67b80642f2ce7e465d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:27.373094 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6d6fb-efb1-43c3-afd7-fcca554f7450 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:27.375799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a8b6d6fb-efb1-43c3-afd7-fcca554f7450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:27.402706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6d6fb-efb1-43c3-afd7-fcca554f7450 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:27.497586 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e9c0e-8b91-48c0-95ae-0c9b9792ba82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:27.498250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e9c0e-8b91-48c0-95ae-0c9b9792ba82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:27.498528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e9c0e-8b91-48c0-95ae-0c9b9792ba82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:27.499005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e9c0e-8b91-48c0-95ae-0c9b9792ba82 admin admin] PATH_INFO: `/v3/projects/de6e36cee9234146abc628e21fdf05ea` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:27.501234 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e9c0e-8b91-48c0-95ae-0c9b9792ba82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de6e36cee9234146abc628e21fdf05ea)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:27.505111 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e9c0e-8b91-48c0-95ae-0c9b9792ba82 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:27.629361 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2e0e9c0e-8b91-48c0-95ae-0c9b9792ba82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de6e36cee9234146abc628e21fdf05ea', 'request_id': 'req-2e0e9c0e-8b91-48c0-95ae-0c9b9792ba82'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:27.640494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2e0e9c0e-8b91-48c0-95ae-0c9b9792ba82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de6e36cee9234146abc628e21fdf05ea', 'request_id': 'req-2e0e9c0e-8b91-48c0-95ae-0c9b9792ba82'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:27.653152 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2e0e9c0e-8b91-48c0-95ae-0c9b9792ba82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de6e36cee9234146abc628e21fdf05ea', 'request_id': 'req-2e0e9c0e-8b91-48c0-95ae-0c9b9792ba82'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:27.689788 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2e0e9c0e-8b91-48c0-95ae-0c9b9792ba82 admin admin] The token cache is being invalidate because project de6e36cee9234146abc628e21fdf05ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:27.690528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2e0e9c0e-8b91-48c0-95ae-0c9b9792ba82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:27.691510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2e0e9c0e-8b91-48c0-95ae-0c9b9792ba82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:27.693931 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 984/1953] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:27 2022] DELETE /identity/v3/projects/de6e36cee9234146abc628e21fdf05ea => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:27.699232 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d593e8-1460-47c8-9ef6-806a9deaef80 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:27.699527 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d593e8-1460-47c8-9ef6-806a9deaef80 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:27.699801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d593e8-1460-47c8-9ef6-806a9deaef80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:27.729334 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-a5d593e8-1460-47c8-9ef6-806a9deaef80 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:27.730636 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a5d593e8-1460-47c8-9ef6-806a9deaef80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:27.864024 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 970/1954] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:15:27.872507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1431bc-a11e-47df-846b-1338071c7fad None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:27.872798 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1431bc-a11e-47df-846b-1338071c7fad None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:27.873119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1431bc-a11e-47df-846b-1338071c7fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:27.897216 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-df1431bc-a11e-47df-846b-1338071c7fad None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:27.898430 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-df1431bc-a11e-47df-846b-1338071c7fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:27.911973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 985/1955] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:27.913455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d593e8-1460-47c8-9ef6-806a9deaef80 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:27.925495 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d593e8-1460-47c8-9ef6-806a9deaef80 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:27.934816 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98c49a-4be2-4096-aae5-a6dda0716afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:27.935472 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98c49a-4be2-4096-aae5-a6dda0716afa admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:27.935765 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98c49a-4be2-4096-aae5-a6dda0716afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:27.936026 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98c49a-4be2-4096-aae5-a6dda0716afa admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:27.936689 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb98c49a-4be2-4096-aae5-a6dda0716afa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:27.938511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb98c49a-4be2-4096-aae5-a6dda0716afa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:27.939032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb98c49a-4be2-4096-aae5-a6dda0716afa admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:27.956882 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 971/1956] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:27.962840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1431bc-a11e-47df-846b-1338071c7fad None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:27.975840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1431bc-a11e-47df-846b-1338071c7fad None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:27.982056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9b9c9-7bcb-4e85-aa5e-60487aad4c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:27.982764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9b9c9-7bcb-4e85-aa5e-60487aad4c1d admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:27.983050 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9b9c9-7bcb-4e85-aa5e-60487aad4c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:27.983368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9b9c9-7bcb-4e85-aa5e-60487aad4c1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:27.984013 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9b9c9-7bcb-4e85-aa5e-60487aad4c1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:27.986742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9b9c9-7bcb-4e85-aa5e-60487aad4c1d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:28.028161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 986/1957] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:27 2022] POST /identity/v3/projects => generated 381 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:15:28.033408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98c49a-4be2-4096-aae5-a6dda0716afa admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:28.044994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98c49a-4be2-4096-aae5-a6dda0716afa admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:28.149746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41238df8-845c-4674-9723-f82daea0ce0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:28.151210 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41238df8-845c-4674-9723-f82daea0ce0f admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:28.152042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41238df8-845c-4674-9723-f82daea0ce0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:28.152529 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41238df8-845c-4674-9723-f82daea0ce0f admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:28.154028 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41238df8-845c-4674-9723-f82daea0ce0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:28.159097 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41238df8-845c-4674-9723-f82daea0ce0f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:28.183876 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:28.210107 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 972/1958] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:28 2022] POST /identity/v3/users => generated 291 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:15:28.215397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9b9c9-7bcb-4e85-aa5e-60487aad4c1d admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:28.237529 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9b9c9-7bcb-4e85-aa5e-60487aad4c1d admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:28.245442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00e5de-3860-415e-83f0-fa26be6b23e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:28.246305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00e5de-3860-415e-83f0-fa26be6b23e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:28.246836 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00e5de-3860-415e-83f0-fa26be6b23e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:28.247328 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00e5de-3860-415e-83f0-fa26be6b23e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:28.248493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00e5de-3860-415e-83f0-fa26be6b23e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:28.251830 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00e5de-3860-415e-83f0-fa26be6b23e1 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:28.272735 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 987/1959] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:28.274247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41238df8-845c-4674-9723-f82daea0ce0f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:28.287208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41238df8-845c-4674-9723-f82daea0ce0f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:28.293793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af4f5c-1b06-429d-96b0-5f3ca76337e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:28.294815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af4f5c-1b06-429d-96b0-5f3ca76337e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:28.295504 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af4f5c-1b06-429d-96b0-5f3ca76337e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:28.296189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af4f5c-1b06-429d-96b0-5f3ca76337e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:28.297650 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af4f5c-1b06-429d-96b0-5f3ca76337e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:28.300672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af4f5c-1b06-429d-96b0-5f3ca76337e7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:28.316337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 973/1960] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:28.321163 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00e5de-3860-415e-83f0-fa26be6b23e1 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:28.334864 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00e5de-3860-415e-83f0-fa26be6b23e1 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:28.340577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681dce9-2a9b-47f0-9e17-91ef3a0308c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:28.341421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681dce9-2a9b-47f0-9e17-91ef3a0308c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:28.341851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681dce9-2a9b-47f0-9e17-91ef3a0308c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:28.342137 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681dce9-2a9b-47f0-9e17-91ef3a0308c6 admin admin] PATH_INFO: `/v3/projects/9486afa53d424bce9b405be21acdbae6/users/544e51814b6b4eb39acd636b9eb0e144/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:28.347456 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:28.384329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681dce9-2a9b-47f0-9e17-91ef3a0308c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9486afa53d424bce9b405be21acdbae6, user_id=544e51814b6b4eb39acd636b9eb0e144, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:28.390194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681dce9-2a9b-47f0-9e17-91ef3a0308c6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:28.417797 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 988/1961] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:28 2022] PUT /identity/v3/projects/9486afa53d424bce9b405be21acdbae6/users/544e51814b6b4eb39acd636b9eb0e144/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:28.445486 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf733b-4faa-42e4-bf29-148f7d06cd2d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:28.445826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf733b-4faa-42e4-bf29-148f7d06cd2d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:28.446059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf733b-4faa-42e4-bf29-148f7d06cd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:28.497666 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-5ccf733b-4faa-42e4-bf29-148f7d06cd2d None None] MFA Rules not processed for user `544e51814b6b4eb39acd636b9eb0e144`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:28.499270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5ccf733b-4faa-42e4-bf29-148f7d06cd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:28.661671 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 974/1962] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:15:28.669774 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681dce9-2a9b-47f0-9e17-91ef3a0308c6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:28.852659 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681dce9-2a9b-47f0-9e17-91ef3a0308c6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:28.862836 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73e329-ea1d-4e86-a3b8-832bef44de4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:28.864450 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73e329-ea1d-4e86-a3b8-832bef44de4d admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:28.864450 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73e329-ea1d-4e86-a3b8-832bef44de4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:28.864797 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73e329-ea1d-4e86-a3b8-832bef44de4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:28.865890 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73e329-ea1d-4e86-a3b8-832bef44de4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:28.870003 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73e329-ea1d-4e86-a3b8-832bef44de4d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:28.945620 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 989/1963] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:28 2022] POST /identity/v3/projects => generated 381 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:15:28.947174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf733b-4faa-42e4-bf29-148f7d06cd2d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:28.961330 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf733b-4faa-42e4-bf29-148f7d06cd2d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:29.044775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1e902-beaf-41c3-ac4d-e80191e7e155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:29.045814 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1e902-beaf-41c3-ac4d-e80191e7e155 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:29.046119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1e902-beaf-41c3-ac4d-e80191e7e155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:29.046372 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1e902-beaf-41c3-ac4d-e80191e7e155 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:29.047074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1e902-beaf-41c3-ac4d-e80191e7e155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:29.049869 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1e902-beaf-41c3-ac4d-e80191e7e155 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:29.075740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 975/1964] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:28 2022] POST /identity/v3/users => generated 297 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:15:29.083290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73e329-ea1d-4e86-a3b8-832bef44de4d admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:29.101426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73e329-ea1d-4e86-a3b8-832bef44de4d admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:29.111775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21878acf-ceb5-4e8b-8195-b1bcbcbceea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:29.112891 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21878acf-ceb5-4e8b-8195-b1bcbcbceea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:29.113374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21878acf-ceb5-4e8b-8195-b1bcbcbceea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:29.113865 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21878acf-ceb5-4e8b-8195-b1bcbcbceea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:29.114982 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21878acf-ceb5-4e8b-8195-b1bcbcbceea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:29.118179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21878acf-ceb5-4e8b-8195-b1bcbcbceea5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:29.139935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 990/1965] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:15:29.147787 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1e902-beaf-41c3-ac4d-e80191e7e155 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:29.168429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1e902-beaf-41c3-ac4d-e80191e7e155 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:29.179729 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c7ec1-d2b9-4010-9f07-badbfff816a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:29.180651 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c7ec1-d2b9-4010-9f07-badbfff816a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:29.181260 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c7ec1-d2b9-4010-9f07-badbfff816a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:29.181694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c7ec1-d2b9-4010-9f07-badbfff816a5 admin admin] PATH_INFO: `/v3/projects/5a48953c77274fc39d80c37eca49a502/users/0c5ff6868ed9470db55784df81d0b5e0/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:29.211635 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c7ec1-d2b9-4010-9f07-badbfff816a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a48953c77274fc39d80c37eca49a502, user_id=0c5ff6868ed9470db55784df81d0b5e0, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:29.215673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c7ec1-d2b9-4010-9f07-badbfff816a5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:29.241832 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 976/1966] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:29 2022] PUT /identity/v3/projects/5a48953c77274fc39d80c37eca49a502/users/0c5ff6868ed9470db55784df81d0b5e0/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:29.248111 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21878acf-ceb5-4e8b-8195-b1bcbcbceea5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:29.262008 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21878acf-ceb5-4e8b-8195-b1bcbcbceea5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:29.305102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c7ec1-d2b9-4010-9f07-badbfff816a5 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:29.307964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-767c7ec1-d2b9-4010-9f07-badbfff816a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:29.397955 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:29.432331 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b5379-fe2d-42f7-9fb2-130ed8ae806b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:29.435204 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b5379-fe2d-42f7-9fb2-130ed8ae806b admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:29.437405 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b5379-fe2d-42f7-9fb2-130ed8ae806b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:29.438327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b5379-fe2d-42f7-9fb2-130ed8ae806b admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:29.440219 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b5379-fe2d-42f7-9fb2-130ed8ae806b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:29.445922 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c7ec1-d2b9-4010-9f07-badbfff816a5 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:29.448247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b5379-fe2d-42f7-9fb2-130ed8ae806b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:29.453119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c62b45-6c7a-4849-96a0-608ec3316fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:29.453941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c62b45-6c7a-4849-96a0-608ec3316fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:29.454285 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c62b45-6c7a-4849-96a0-608ec3316fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:29.454739 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c62b45-6c7a-4849-96a0-608ec3316fdd admin admin] PATH_INFO: `/v3/users/0eaa3705e7a0489a9c0b94fbf69d08eb` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:29.456428 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c62b45-6c7a-4849-96a0-608ec3316fdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eaa3705e7a0489a9c0b94fbf69d08eb)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:29.459055 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c62b45-6c7a-4849-96a0-608ec3316fdd admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:29.475470 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 991/1967] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:15:29.480304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b5379-fe2d-42f7-9fb2-130ed8ae806b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:29.507338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b5379-fe2d-42f7-9fb2-130ed8ae806b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:29.526830 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709e3be-91a1-471f-9e71-1fe4d22dd481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:29.527864 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709e3be-91a1-471f-9e71-1fe4d22dd481 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:29.528301 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709e3be-91a1-471f-9e71-1fe4d22dd481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:29.528724 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709e3be-91a1-471f-9e71-1fe4d22dd481 admin admin] PATH_INFO: `/v3/projects/5a48953c77274fc39d80c37eca49a502/users/0c5ff6868ed9470db55784df81d0b5e0/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:29.531359 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709e3be-91a1-471f-9e71-1fe4d22dd481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a48953c77274fc39d80c37eca49a502, user_id=0c5ff6868ed9470db55784df81d0b5e0, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:29.533267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-02c62b45-6c7a-4849-96a0-608ec3316fdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eaa3705e7a0489a9c0b94fbf69d08eb'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:29.537023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709e3be-91a1-471f-9e71-1fe4d22dd481 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:29.545125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-02c62b45-6c7a-4849-96a0-608ec3316fdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eaa3705e7a0489a9c0b94fbf69d08eb'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:29.554638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 992/1968] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:29 2022] PUT /identity/v3/projects/5a48953c77274fc39d80c37eca49a502/users/0c5ff6868ed9470db55784df81d0b5e0/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:29.574391 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37db709-6294-4d07-8890-3349f1a03ad5 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:29.575172 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37db709-6294-4d07-8890-3349f1a03ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:29.576008 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37db709-6294-4d07-8890-3349f1a03ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:29.597418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-02c62b45-6c7a-4849-96a0-608ec3316fdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eaa3705e7a0489a9c0b94fbf69d08eb'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:29.602969 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-02c62b45-6c7a-4849-96a0-608ec3316fdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eaa3705e7a0489a9c0b94fbf69d08eb'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:29.603997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-02c62b45-6c7a-4849-96a0-608ec3316fdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eaa3705e7a0489a9c0b94fbf69d08eb'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:29.629808 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 977/1969] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:29 2022] DELETE /identity/v3/users/0eaa3705e7a0489a9c0b94fbf69d08eb => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:29.637701 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a37db709-6294-4d07-8890-3349f1a03ad5 None None] MFA Rules not processed for user `0c5ff6868ed9470db55784df81d0b5e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:29.654966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800381c4-93dd-432f-9c4a-9b0aa43a6cce None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:29.654966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800381c4-93dd-432f-9c4a-9b0aa43a6cce None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:29.654966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800381c4-93dd-432f-9c4a-9b0aa43a6cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:29.655447 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a37db709-6294-4d07-8890-3349f1a03ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:29.693399 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-800381c4-93dd-432f-9c4a-9b0aa43a6cce None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:29.696227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-800381c4-93dd-432f-9c4a-9b0aa43a6cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:29.700059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:29.758558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 993/1970] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:15:29.781216 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37db709-6294-4d07-8890-3349f1a03ad5 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:29.783250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a37db709-6294-4d07-8890-3349f1a03ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:29.841363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 978/1971] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:29.858997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800381c4-93dd-432f-9c4a-9b0aa43a6cce None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:29.866652 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-800381c4-93dd-432f-9c4a-9b0aa43a6cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:29.928764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37db709-6294-4d07-8890-3349f1a03ad5 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:29.935405 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca0846-1425-4a7a-9704-193fe0d04b92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:29.936195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca0846-1425-4a7a-9704-193fe0d04b92 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:29.937191 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca0846-1425-4a7a-9704-193fe0d04b92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:29.938026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca0846-1425-4a7a-9704-193fe0d04b92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:29.954374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cca0846-1425-4a7a-9704-193fe0d04b92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:29.957543 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cca0846-1425-4a7a-9704-193fe0d04b92 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:29.969784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800381c4-93dd-432f-9c4a-9b0aa43a6cce None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:29.976049 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f7969-f40f-4bad-8c44-a2f4b61786fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:29.977020 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f7969-f40f-4bad-8c44-a2f4b61786fc service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:29.977757 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f7969-f40f-4bad-8c44-a2f4b61786fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:29.978289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f7969-f40f-4bad-8c44-a2f4b61786fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:29.991461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 994/1972] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:29.992433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:29.996955 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f7969-f40f-4bad-8c44-a2f4b61786fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:30.000734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f7969-f40f-4bad-8c44-a2f4b61786fc service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:30.024077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 979/1973] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:15:30.553168 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca0846-1425-4a7a-9704-193fe0d04b92 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:30.557164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1cca0846-1425-4a7a-9704-193fe0d04b92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:30.615881 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca0846-1425-4a7a-9704-193fe0d04b92 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:30.632474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1debdfc-f048-4f0f-987e-aabe554b14f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:30.636913 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1debdfc-f048-4f0f-987e-aabe554b14f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:30.640383 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1debdfc-f048-4f0f-987e-aabe554b14f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:30.641125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1debdfc-f048-4f0f-987e-aabe554b14f6 admin admin] PATH_INFO: `/v3/projects/1271e568c39841bdba343e6ac04bab20` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:30.653136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1debdfc-f048-4f0f-987e-aabe554b14f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1271e568c39841bdba343e6ac04bab20)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:30.663309 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1debdfc-f048-4f0f-987e-aabe554b14f6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:30.700830 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f7969-f40f-4bad-8c44-a2f4b61786fc service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:30.704265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-353f7969-f40f-4bad-8c44-a2f4b61786fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:30.765033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f7969-f40f-4bad-8c44-a2f4b61786fc service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:30.791336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34064038-ab97-45eb-b653-c63b3c8d45e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:30.793273 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34064038-ab97-45eb-b653-c63b3c8d45e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:30.793559 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34064038-ab97-45eb-b653-c63b3c8d45e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:30.794044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34064038-ab97-45eb-b653-c63b3c8d45e4 admin admin] PATH_INFO: `/v3/users/544e51814b6b4eb39acd636b9eb0e144` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:30.798741 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34064038-ab97-45eb-b653-c63b3c8d45e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=544e51814b6b4eb39acd636b9eb0e144)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:30.806476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34064038-ab97-45eb-b653-c63b3c8d45e4 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:30.820092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d1debdfc-f048-4f0f-987e-aabe554b14f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1271e568c39841bdba343e6ac04bab20', 'request_id': 'req-d1debdfc-f048-4f0f-987e-aabe554b14f6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:30.846653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d1debdfc-f048-4f0f-987e-aabe554b14f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1271e568c39841bdba343e6ac04bab20', 'request_id': 'req-d1debdfc-f048-4f0f-987e-aabe554b14f6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:30.866000 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d1debdfc-f048-4f0f-987e-aabe554b14f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1271e568c39841bdba343e6ac04bab20', 'request_id': 'req-d1debdfc-f048-4f0f-987e-aabe554b14f6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:30.913215 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d1debdfc-f048-4f0f-987e-aabe554b14f6 admin admin] The token cache is being invalidate because project 1271e568c39841bdba343e6ac04bab20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:30.913696 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d1debdfc-f048-4f0f-987e-aabe554b14f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:30.914742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d1debdfc-f048-4f0f-987e-aabe554b14f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:30.917599 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 995/1974] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:30 2022] DELETE /identity/v3/projects/1271e568c39841bdba343e6ac04bab20 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:30.923777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1debdfc-f048-4f0f-987e-aabe554b14f6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:30.928661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d1debdfc-f048-4f0f-987e-aabe554b14f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:30.929039 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-34064038-ab97-45eb-b653-c63b3c8d45e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '544e51814b6b4eb39acd636b9eb0e144'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:30.944784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-34064038-ab97-45eb-b653-c63b3c8d45e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '544e51814b6b4eb39acd636b9eb0e144'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:30.973283 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-34064038-ab97-45eb-b653-c63b3c8d45e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '544e51814b6b4eb39acd636b9eb0e144'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:30.974505 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-34064038-ab97-45eb-b653-c63b3c8d45e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '544e51814b6b4eb39acd636b9eb0e144'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:30.975498 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-34064038-ab97-45eb-b653-c63b3c8d45e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '544e51814b6b4eb39acd636b9eb0e144'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:31.000758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 980/1975] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:30 2022] DELETE /identity/v3/users/544e51814b6b4eb39acd636b9eb0e144 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:31.008510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1ee4c-0c74-413e-9fc5-a1128a7bdd2c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:31.009804 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1ee4c-0c74-413e-9fc5-a1128a7bdd2c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:31.010696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1ee4c-0c74-413e-9fc5-a1128a7bdd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:31.078542 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:31.083480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1ae1ee4c-0c74-413e-9fc5-a1128a7bdd2c None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:31.093143 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1ae1ee4c-0c74-413e-9fc5-a1128a7bdd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:31.123535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1debdfc-f048-4f0f-987e-aabe554b14f6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:31.313966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc363e3c-edd3-4717-a84a-3659b95ec9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:31.315819 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc363e3c-edd3-4717-a84a-3659b95ec9f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:31.316218 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc363e3c-edd3-4717-a84a-3659b95ec9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:31.316693 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc363e3c-edd3-4717-a84a-3659b95ec9f3 admin admin] PATH_INFO: `/v3/users/c17ccfaf8cc04caa9b9242123dbb958b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:31.325570 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc363e3c-edd3-4717-a84a-3659b95ec9f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c17ccfaf8cc04caa9b9242123dbb958b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:31.333545 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 981/1976] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:31.346226 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1ee4c-0c74-413e-9fc5-a1128a7bdd2c None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:31.346534 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc363e3c-edd3-4717-a84a-3659b95ec9f3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:31.348400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1ae1ee4c-0c74-413e-9fc5-a1128a7bdd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:31.542734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1ee4c-0c74-413e-9fc5-a1128a7bdd2c None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:31.558072 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf93a9-9aa1-449b-881a-3c3b26a09f06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:31.558743 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf93a9-9aa1-449b-881a-3c3b26a09f06 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:31.559024 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf93a9-9aa1-449b-881a-3c3b26a09f06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:31.559298 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf93a9-9aa1-449b-881a-3c3b26a09f06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:31.594221 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bc363e3c-edd3-4717-a84a-3659b95ec9f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c17ccfaf8cc04caa9b9242123dbb958b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:31.608688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf93a9-9aa1-449b-881a-3c3b26a09f06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:31.610175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf93a9-9aa1-449b-881a-3c3b26a09f06 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:31.640689 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bc363e3c-edd3-4717-a84a-3659b95ec9f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c17ccfaf8cc04caa9b9242123dbb958b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:31.644616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 982/1977] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:31.693459 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bc363e3c-edd3-4717-a84a-3659b95ec9f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c17ccfaf8cc04caa9b9242123dbb958b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:31.694813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bc363e3c-edd3-4717-a84a-3659b95ec9f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c17ccfaf8cc04caa9b9242123dbb958b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:31.717782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bc363e3c-edd3-4717-a84a-3659b95ec9f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c17ccfaf8cc04caa9b9242123dbb958b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:31.734730 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 996/1978] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:30 2022] DELETE /identity/v3/users/c17ccfaf8cc04caa9b9242123dbb958b => generated 0 bytes in 812 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:32.159478 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf93a9-9aa1-449b-881a-3c3b26a09f06 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:32.164331 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7cdf93a9-9aa1-449b-881a-3c3b26a09f06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:32.206035 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf93a9-9aa1-449b-881a-3c3b26a09f06 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:32.234120 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:32.404442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61887bb8-2d58-47d6-8be2-dcfd111f08d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:32.404442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61887bb8-2d58-47d6-8be2-dcfd111f08d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:32.404861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61887bb8-2d58-47d6-8be2-dcfd111f08d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:32.404861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61887bb8-2d58-47d6-8be2-dcfd111f08d2 admin admin] PATH_INFO: `/v3/users/2f167e0c4fe6494daafbc8d607656d10` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:32.409234 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61887bb8-2d58-47d6-8be2-dcfd111f08d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f167e0c4fe6494daafbc8d607656d10)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:32.411398 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61887bb8-2d58-47d6-8be2-dcfd111f08d2 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:32.550991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-61887bb8-2d58-47d6-8be2-dcfd111f08d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f167e0c4fe6494daafbc8d607656d10'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:32.579136 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-61887bb8-2d58-47d6-8be2-dcfd111f08d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f167e0c4fe6494daafbc8d607656d10'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:32.612575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-61887bb8-2d58-47d6-8be2-dcfd111f08d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f167e0c4fe6494daafbc8d607656d10'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:32.613590 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-61887bb8-2d58-47d6-8be2-dcfd111f08d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f167e0c4fe6494daafbc8d607656d10'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:32.614357 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-61887bb8-2d58-47d6-8be2-dcfd111f08d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f167e0c4fe6494daafbc8d607656d10'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:32.654457 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065ff8c-c8d4-4a66-8e08-04be53072326 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:32.655098 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 983/1979] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:32 2022] DELETE /identity/v3/users/2f167e0c4fe6494daafbc8d607656d10 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:32.655197 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065ff8c-c8d4-4a66-8e08-04be53072326 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:32.655197 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065ff8c-c8d4-4a66-8e08-04be53072326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:32.724947 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8065ff8c-c8d4-4a66-8e08-04be53072326 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:32.732007 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8065ff8c-c8d4-4a66-8e08-04be53072326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:32.867301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61887bb8-2d58-47d6-8be2-dcfd111f08d2 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:32.868692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 997/1980] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:15:32.874706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-61887bb8-2d58-47d6-8be2-dcfd111f08d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:32.900125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61887bb8-2d58-47d6-8be2-dcfd111f08d2 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:32.904686 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065ff8c-c8d4-4a66-8e08-04be53072326 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:32.907150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8065ff8c-c8d4-4a66-8e08-04be53072326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:32.913036 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923a523-8f9a-4f12-8dbb-309d36f7cf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:32.917055 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923a523-8f9a-4f12-8dbb-309d36f7cf76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:32.917055 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923a523-8f9a-4f12-8dbb-309d36f7cf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:32.917439 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923a523-8f9a-4f12-8dbb-309d36f7cf76 admin admin] PATH_INFO: `/v3/projects/d9ff96d9dec5461f900b8fd73d4f5462` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:32.923968 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8923a523-8f9a-4f12-8dbb-309d36f7cf76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ff96d9dec5461f900b8fd73d4f5462)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:32.929572 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8923a523-8f9a-4f12-8dbb-309d36f7cf76 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:32.948728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:33.032668 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065ff8c-c8d4-4a66-8e08-04be53072326 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:33.040004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8923a523-8f9a-4f12-8dbb-309d36f7cf76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ff96d9dec5461f900b8fd73d4f5462', 'request_id': 'req-8923a523-8f9a-4f12-8dbb-309d36f7cf76'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:33.050835 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8923a523-8f9a-4f12-8dbb-309d36f7cf76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ff96d9dec5461f900b8fd73d4f5462', 'request_id': 'req-8923a523-8f9a-4f12-8dbb-309d36f7cf76'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:33.073341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8923a523-8f9a-4f12-8dbb-309d36f7cf76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ff96d9dec5461f900b8fd73d4f5462', 'request_id': 'req-8923a523-8f9a-4f12-8dbb-309d36f7cf76'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:33.100888 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8923a523-8f9a-4f12-8dbb-309d36f7cf76 admin admin] The token cache is being invalidate because project d9ff96d9dec5461f900b8fd73d4f5462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:33.101360 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8923a523-8f9a-4f12-8dbb-309d36f7cf76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:33.104924 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8923a523-8f9a-4f12-8dbb-309d36f7cf76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:33.109811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 984/1981] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:32 2022] DELETE /identity/v3/projects/d9ff96d9dec5461f900b8fd73d4f5462 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:33.113673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923a523-8f9a-4f12-8dbb-309d36f7cf76 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:33.116449 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8923a523-8f9a-4f12-8dbb-309d36f7cf76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:33.160986 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:33.195199 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf62e0-06c5-452e-a54a-a296031b2201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:33.197258 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf62e0-06c5-452e-a54a-a296031b2201 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:33.198048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf62e0-06c5-452e-a54a-a296031b2201 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:33.198816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf62e0-06c5-452e-a54a-a296031b2201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:33.202373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-91cf62e0-06c5-452e-a54a-a296031b2201 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:33.290254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923a523-8f9a-4f12-8dbb-309d36f7cf76 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:33.300073 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910c9fb-6c56-4dfb-8d5f-a50c4b640ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:33.300912 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910c9fb-6c56-4dfb-8d5f-a50c4b640ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:33.301382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910c9fb-6c56-4dfb-8d5f-a50c4b640ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:33.302107 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910c9fb-6c56-4dfb-8d5f-a50c4b640ecf admin admin] PATH_INFO: `/v3/projects/9486afa53d424bce9b405be21acdbae6` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:33.304459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910c9fb-6c56-4dfb-8d5f-a50c4b640ecf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9486afa53d424bce9b405be21acdbae6)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:33.309590 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910c9fb-6c56-4dfb-8d5f-a50c4b640ecf admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:33.339059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf62e0-06c5-452e-a54a-a296031b2201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:33.340752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf62e0-06c5-452e-a54a-a296031b2201 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:33.354708 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 998/1982] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:15:33.355833 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf62e0-06c5-452e-a54a-a296031b2201 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:33.357807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-91cf62e0-06c5-452e-a54a-a296031b2201 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:33.384585 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf62e0-06c5-452e-a54a-a296031b2201 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:33.446425 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f910c9fb-6c56-4dfb-8d5f-a50c4b640ecf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9486afa53d424bce9b405be21acdbae6', 'request_id': 'req-f910c9fb-6c56-4dfb-8d5f-a50c4b640ecf'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:33.471185 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f910c9fb-6c56-4dfb-8d5f-a50c4b640ecf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9486afa53d424bce9b405be21acdbae6', 'request_id': 'req-f910c9fb-6c56-4dfb-8d5f-a50c4b640ecf'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:33.496401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f910c9fb-6c56-4dfb-8d5f-a50c4b640ecf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9486afa53d424bce9b405be21acdbae6', 'request_id': 'req-f910c9fb-6c56-4dfb-8d5f-a50c4b640ecf'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:33.553472 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f910c9fb-6c56-4dfb-8d5f-a50c4b640ecf admin admin] The token cache is being invalidate because project 9486afa53d424bce9b405be21acdbae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:33.553999 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f910c9fb-6c56-4dfb-8d5f-a50c4b640ecf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:33.557869 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f910c9fb-6c56-4dfb-8d5f-a50c4b640ecf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:33.567305 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 985/1983] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:33 2022] DELETE /identity/v3/projects/9486afa53d424bce9b405be21acdbae6 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:33.574550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910c9fb-6c56-4dfb-8d5f-a50c4b640ecf admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:33.588192 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f910c9fb-6c56-4dfb-8d5f-a50c4b640ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:33.629121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45dc43-11ab-4827-bf96-45012c4f2ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:33.633049 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45dc43-11ab-4827-bf96-45012c4f2ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:33.633049 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45dc43-11ab-4827-bf96-45012c4f2ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:33.633049 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45dc43-11ab-4827-bf96-45012c4f2ce4 admin admin] PATH_INFO: `/v3/users/88205bbb257a4b34939629a35fbfbd64` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:33.637375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45dc43-11ab-4827-bf96-45012c4f2ce4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88205bbb257a4b34939629a35fbfbd64)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:33.640847 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45dc43-11ab-4827-bf96-45012c4f2ce4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:33.724324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:33.816176 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ea45dc43-11ab-4827-bf96-45012c4f2ce4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88205bbb257a4b34939629a35fbfbd64'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:33.835871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910c9fb-6c56-4dfb-8d5f-a50c4b640ecf admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:33.848737 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ea45dc43-11ab-4827-bf96-45012c4f2ce4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88205bbb257a4b34939629a35fbfbd64'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:33.863097 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ff601-e85f-4477-9776-62ba695a5246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:33.864317 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ff601-e85f-4477-9776-62ba695a5246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:33.864833 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ff601-e85f-4477-9776-62ba695a5246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:33.865320 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ff601-e85f-4477-9776-62ba695a5246 admin admin] PATH_INFO: `/v3/users/0c5ff6868ed9470db55784df81d0b5e0` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:33.870530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ff601-e85f-4477-9776-62ba695a5246 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c5ff6868ed9470db55784df81d0b5e0)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:33.885711 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ff601-e85f-4477-9776-62ba695a5246 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:33.910866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ea45dc43-11ab-4827-bf96-45012c4f2ce4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88205bbb257a4b34939629a35fbfbd64'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:33.914448 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ea45dc43-11ab-4827-bf96-45012c4f2ce4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88205bbb257a4b34939629a35fbfbd64'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:33.942345 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ea45dc43-11ab-4827-bf96-45012c4f2ce4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88205bbb257a4b34939629a35fbfbd64'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:33.971039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 999/1984] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:33 2022] DELETE /identity/v3/users/88205bbb257a4b34939629a35fbfbd64 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:34.150170 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:34.161378 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e26ff601-e85f-4477-9776-62ba695a5246 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c5ff6868ed9470db55784df81d0b5e0'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:34.213345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e26ff601-e85f-4477-9776-62ba695a5246 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c5ff6868ed9470db55784df81d0b5e0'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:34.302685 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e26ff601-e85f-4477-9776-62ba695a5246 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c5ff6868ed9470db55784df81d0b5e0'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:34.304669 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e26ff601-e85f-4477-9776-62ba695a5246 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c5ff6868ed9470db55784df81d0b5e0'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:34.306216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e26ff601-e85f-4477-9776-62ba695a5246 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c5ff6868ed9470db55784df81d0b5e0'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:34.405804 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 986/1985] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:33 2022] DELETE /identity/v3/users/0c5ff6868ed9470db55784df81d0b5e0 => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:34.803159 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45dc43-11ab-4827-bf96-45012c4f2ce4 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:34.804794 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ea45dc43-11ab-4827-bf96-45012c4f2ce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:34.998204 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45dc43-11ab-4827-bf96-45012c4f2ce4 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:35.132744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c327c-1cc6-4a07-bc59-0ec34b7538dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:35.133848 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c327c-1cc6-4a07-bc59-0ec34b7538dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:35.134350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c327c-1cc6-4a07-bc59-0ec34b7538dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:35.134903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c327c-1cc6-4a07-bc59-0ec34b7538dd admin admin] PATH_INFO: `/v3/projects/5b7d99d91ca94bca814f7c9e7b39b0a4` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:35.137827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c327c-1cc6-4a07-bc59-0ec34b7538dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b7d99d91ca94bca814f7c9e7b39b0a4)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:35.141798 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c327c-1cc6-4a07-bc59-0ec34b7538dd admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:35.166844 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ff601-e85f-4477-9776-62ba695a5246 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:35.170392 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e26ff601-e85f-4477-9776-62ba695a5246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:35.237269 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ff601-e85f-4477-9776-62ba695a5246 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:35.251403 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719cea6-d938-465a-9fa8-970ade713ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:35.253010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719cea6-d938-465a-9fa8-970ade713ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:35.253853 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719cea6-d938-465a-9fa8-970ade713ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:35.256512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719cea6-d938-465a-9fa8-970ade713ba6 admin admin] PATH_INFO: `/v3/projects/ac9adc7708ab4dd0a967ca465bab86ed` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:35.259763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6719cea6-d938-465a-9fa8-970ade713ba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac9adc7708ab4dd0a967ca465bab86ed)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:35.264466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6719cea6-d938-465a-9fa8-970ade713ba6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:35.300572 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8d1c327c-1cc6-4a07-bc59-0ec34b7538dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b7d99d91ca94bca814f7c9e7b39b0a4', 'request_id': 'req-8d1c327c-1cc6-4a07-bc59-0ec34b7538dd'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:35.305433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:35.315748 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8d1c327c-1cc6-4a07-bc59-0ec34b7538dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b7d99d91ca94bca814f7c9e7b39b0a4', 'request_id': 'req-8d1c327c-1cc6-4a07-bc59-0ec34b7538dd'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:35.328465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8d1c327c-1cc6-4a07-bc59-0ec34b7538dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b7d99d91ca94bca814f7c9e7b39b0a4', 'request_id': 'req-8d1c327c-1cc6-4a07-bc59-0ec34b7538dd'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:35.359516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8d1c327c-1cc6-4a07-bc59-0ec34b7538dd admin admin] The token cache is being invalidate because project 5b7d99d91ca94bca814f7c9e7b39b0a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:35.361312 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8d1c327c-1cc6-4a07-bc59-0ec34b7538dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:35.362484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8d1c327c-1cc6-4a07-bc59-0ec34b7538dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:35.367561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1000/1986] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:34 2022] DELETE /identity/v3/projects/5b7d99d91ca94bca814f7c9e7b39b0a4 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:35.371686 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c327c-1cc6-4a07-bc59-0ec34b7538dd admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:35.374037 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8d1c327c-1cc6-4a07-bc59-0ec34b7538dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:35.407395 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-6719cea6-d938-465a-9fa8-970ade713ba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac9adc7708ab4dd0a967ca465bab86ed', 'request_id': 'req-6719cea6-d938-465a-9fa8-970ade713ba6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:35.421059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-6719cea6-d938-465a-9fa8-970ade713ba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac9adc7708ab4dd0a967ca465bab86ed', 'request_id': 'req-6719cea6-d938-465a-9fa8-970ade713ba6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:35.433396 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-6719cea6-d938-465a-9fa8-970ade713ba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac9adc7708ab4dd0a967ca465bab86ed', 'request_id': 'req-6719cea6-d938-465a-9fa8-970ade713ba6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:35.451169 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-6719cea6-d938-465a-9fa8-970ade713ba6 admin admin] The token cache is being invalidate because project ac9adc7708ab4dd0a967ca465bab86ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:35.451712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-6719cea6-d938-465a-9fa8-970ade713ba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:35.452461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-6719cea6-d938-465a-9fa8-970ade713ba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:35.455363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 987/1987] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:35 2022] DELETE /identity/v3/projects/ac9adc7708ab4dd0a967ca465bab86ed => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:35.463958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8138569-a76d-4587-b8f1-437d356ab095 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:35.464423 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8138569-a76d-4587-b8f1-437d356ab095 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:35.464851 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8138569-a76d-4587-b8f1-437d356ab095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:35.495554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-c8138569-a76d-4587-b8f1-437d356ab095 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:35.497164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c327c-1cc6-4a07-bc59-0ec34b7538dd admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:35.498482 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c8138569-a76d-4587-b8f1-437d356ab095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:35.629591 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4d263-161b-43ac-9433-75c090f2304f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:35.631740 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4d263-161b-43ac-9433-75c090f2304f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:35.632658 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4d263-161b-43ac-9433-75c090f2304f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:35.634832 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4d263-161b-43ac-9433-75c090f2304f admin admin] PATH_INFO: `/v3/users/30f851e0460f408981dc139d502a4647` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:35.638961 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4d263-161b-43ac-9433-75c090f2304f admin admin] RBAC: Authorizing `identity:delete_user(user_id=30f851e0460f408981dc139d502a4647)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:35.646057 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4d263-161b-43ac-9433-75c090f2304f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:35.676971 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 988/1988] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:35.680740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98ca8b-8e6a-4143-8198-a3dfee3c429a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:35.681107 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98ca8b-8e6a-4143-8198-a3dfee3c429a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:35.681459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98ca8b-8e6a-4143-8198-a3dfee3c429a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:35.714717 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8a98ca8b-8e6a-4143-8198-a3dfee3c429a None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:35.716496 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8a98ca8b-8e6a-4143-8198-a3dfee3c429a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:35.744756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 989/1989] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:35.757923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98ca8b-8e6a-4143-8198-a3dfee3c429a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:35.770147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-25b4d263-161b-43ac-9433-75c090f2304f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30f851e0460f408981dc139d502a4647'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:35.773061 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98ca8b-8e6a-4143-8198-a3dfee3c429a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:35.779526 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160349d9-1014-49ad-8ab9-bbd80e4da150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:35.780086 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160349d9-1014-49ad-8ab9-bbd80e4da150 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:35.780368 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160349d9-1014-49ad-8ab9-bbd80e4da150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:35.780636 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160349d9-1014-49ad-8ab9-bbd80e4da150 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:35.781793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160349d9-1014-49ad-8ab9-bbd80e4da150 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:35.782103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160349d9-1014-49ad-8ab9-bbd80e4da150 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:35.784782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160349d9-1014-49ad-8ab9-bbd80e4da150 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:35.789058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-25b4d263-161b-43ac-9433-75c090f2304f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30f851e0460f408981dc139d502a4647'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:35.804032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 990/1990] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:35.808648 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160349d9-1014-49ad-8ab9-bbd80e4da150 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:35.809024 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-160349d9-1014-49ad-8ab9-bbd80e4da150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:35.842548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160349d9-1014-49ad-8ab9-bbd80e4da150 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:35.850150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-25b4d263-161b-43ac-9433-75c090f2304f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30f851e0460f408981dc139d502a4647'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:35.851018 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-25b4d263-161b-43ac-9433-75c090f2304f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30f851e0460f408981dc139d502a4647'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:35.854484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13f17b-6994-403e-a57f-1af6f0d8f70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:35.855769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13f17b-6994-403e-a57f-1af6f0d8f70a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:35.856032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13f17b-6994-403e-a57f-1af6f0d8f70a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:35.856305 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13f17b-6994-403e-a57f-1af6f0d8f70a admin admin] PATH_INFO: `/v3/projects/5a48953c77274fc39d80c37eca49a502` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:35.858030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13f17b-6994-403e-a57f-1af6f0d8f70a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a48953c77274fc39d80c37eca49a502)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:35.860488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13f17b-6994-403e-a57f-1af6f0d8f70a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:35.866902 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-25b4d263-161b-43ac-9433-75c090f2304f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30f851e0460f408981dc139d502a4647'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:35.877147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1001/1991] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:35 2022] DELETE /identity/v3/users/30f851e0460f408981dc139d502a4647 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:35.877690 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4d263-161b-43ac-9433-75c090f2304f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:35.881397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-25b4d263-161b-43ac-9433-75c090f2304f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:35.895045 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:35.897484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:35.961158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fe13f17b-6994-403e-a57f-1af6f0d8f70a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a48953c77274fc39d80c37eca49a502', 'request_id': 'req-fe13f17b-6994-403e-a57f-1af6f0d8f70a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:35.976133 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fe13f17b-6994-403e-a57f-1af6f0d8f70a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a48953c77274fc39d80c37eca49a502', 'request_id': 'req-fe13f17b-6994-403e-a57f-1af6f0d8f70a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:36.015396 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fe13f17b-6994-403e-a57f-1af6f0d8f70a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a48953c77274fc39d80c37eca49a502', 'request_id': 'req-fe13f17b-6994-403e-a57f-1af6f0d8f70a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:36.050525 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4d263-161b-43ac-9433-75c090f2304f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:36.059785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fe13f17b-6994-403e-a57f-1af6f0d8f70a admin admin] The token cache is being invalidate because project 5a48953c77274fc39d80c37eca49a502 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:36.061908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fe13f17b-6994-403e-a57f-1af6f0d8f70a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:36.063718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fe13f17b-6994-403e-a57f-1af6f0d8f70a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:36.081280 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 991/1992] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:35 2022] DELETE /identity/v3/projects/5a48953c77274fc39d80c37eca49a502 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:36.084525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca2f89-2768-463b-8030-b48949988019 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:36.086007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca2f89-2768-463b-8030-b48949988019 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:36.086970 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca2f89-2768-463b-8030-b48949988019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:36.135372 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-fdca2f89-2768-463b-8030-b48949988019 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:36.139051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fdca2f89-2768-463b-8030-b48949988019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:36.235292 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1e58b-ecf3-4cb7-8833-62d0e0f3d1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:36.236809 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1e58b-ecf3-4cb7-8833-62d0e0f3d1cd admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:36.237413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1e58b-ecf3-4cb7-8833-62d0e0f3d1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:36.240923 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1e58b-ecf3-4cb7-8833-62d0e0f3d1cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:36.242730 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1e58b-ecf3-4cb7-8833-62d0e0f3d1cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:36.248131 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1e58b-ecf3-4cb7-8833-62d0e0f3d1cd admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:36.355436 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 992/1993] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:36.365920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87f6d3-e910-4465-a7ee-17be91d23915 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:36.366374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87f6d3-e910-4465-a7ee-17be91d23915 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:36.366891 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87f6d3-e910-4465-a7ee-17be91d23915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:36.368999 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1002/1994] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:35 2022] POST /identity/v3/projects => generated 381 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:36.374327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1e58b-ecf3-4cb7-8833-62d0e0f3d1cd admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:36.380014 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4ea1e58b-ecf3-4cb7-8833-62d0e0f3d1cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:36.432457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-7a87f6d3-e910-4465-a7ee-17be91d23915 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:36.433894 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7a87f6d3-e910-4465-a7ee-17be91d23915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:36.559715 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 993/1995] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:15:36.565263 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87f6d3-e910-4465-a7ee-17be91d23915 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:36.582480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87f6d3-e910-4465-a7ee-17be91d23915 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:36.587691 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05506c8c-ee2f-4760-8473-5496cf6dbae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:36.588318 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05506c8c-ee2f-4760-8473-5496cf6dbae1 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:36.588606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05506c8c-ee2f-4760-8473-5496cf6dbae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:36.588856 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05506c8c-ee2f-4760-8473-5496cf6dbae1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:36.589699 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05506c8c-ee2f-4760-8473-5496cf6dbae1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:36.590002 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05506c8c-ee2f-4760-8473-5496cf6dbae1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:36.591709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05506c8c-ee2f-4760-8473-5496cf6dbae1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:36.621352 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1e58b-ecf3-4cb7-8833-62d0e0f3d1cd admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:36.633676 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afc2ad-7eb7-44d2-989b-fa1224b3738f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:36.635133 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 994/1996] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:15:36.635365 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afc2ad-7eb7-44d2-989b-fa1224b3738f admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:36.635998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afc2ad-7eb7-44d2-989b-fa1224b3738f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:36.636141 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05506c8c-ee2f-4760-8473-5496cf6dbae1 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:36.636734 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afc2ad-7eb7-44d2-989b-fa1224b3738f admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:36.638377 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4afc2ad-7eb7-44d2-989b-fa1224b3738f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:36.643847 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4afc2ad-7eb7-44d2-989b-fa1224b3738f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:36.674147 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05506c8c-ee2f-4760-8473-5496cf6dbae1 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:36.696662 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1003/1997] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:36 2022] POST /identity/v3/users => generated 291 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:36.697661 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527b041-9ba6-4c67-8654-af59ec0ede34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:36.698328 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527b041-9ba6-4c67-8654-af59ec0ede34 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:36.698788 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527b041-9ba6-4c67-8654-af59ec0ede34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:36.699094 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527b041-9ba6-4c67-8654-af59ec0ede34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:36.700126 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527b041-9ba6-4c67-8654-af59ec0ede34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:36.703672 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afc2ad-7eb7-44d2-989b-fa1224b3738f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:36.706271 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527b041-9ba6-4c67-8654-af59ec0ede34 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:36.754390 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afc2ad-7eb7-44d2-989b-fa1224b3738f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:36.768358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266b84a-6b03-46e1-a252-d351982c55d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:36.770501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266b84a-6b03-46e1-a252-d351982c55d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:36.770857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266b84a-6b03-46e1-a252-d351982c55d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:36.772055 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266b84a-6b03-46e1-a252-d351982c55d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:36.773888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266b84a-6b03-46e1-a252-d351982c55d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:36.776811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 995/1998] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:36 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:36.778839 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527b041-9ba6-4c67-8654-af59ec0ede34 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:36.781031 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b527b041-9ba6-4c67-8654-af59ec0ede34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:36.781331 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266b84a-6b03-46e1-a252-d351982c55d4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:36.820934 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1004/1999] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:36.822556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266b84a-6b03-46e1-a252-d351982c55d4 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:36.828093 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0266b84a-6b03-46e1-a252-d351982c55d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:36.851249 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:36.873407 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266b84a-6b03-46e1-a252-d351982c55d4 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:36.904629 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:36.941123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527b041-9ba6-4c67-8654-af59ec0ede34 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:36.947268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b97891-01b4-44ef-a259-38260a2453ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:36.948202 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b97891-01b4-44ef-a259-38260a2453ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:36.948483 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b97891-01b4-44ef-a259-38260a2453ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:36.948763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b97891-01b4-44ef-a259-38260a2453ec admin admin] PATH_INFO: `/v3/projects/71e74ae793254faeafcae5fffff258e9` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:36.949989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b97891-01b4-44ef-a259-38260a2453ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=71e74ae793254faeafcae5fffff258e9)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:36.951999 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b97891-01b4-44ef-a259-38260a2453ec admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:36.987720 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2e197-d33b-4bed-9415-5836fc50b065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:36.989566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2e197-d33b-4bed-9415-5836fc50b065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:36.990040 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2e197-d33b-4bed-9415-5836fc50b065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:36.990482 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2e197-d33b-4bed-9415-5836fc50b065 admin admin] PATH_INFO: `/v3/users/fbf321d4bfbc4f248c79ac584554c00b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:36.992668 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2e197-d33b-4bed-9415-5836fc50b065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbf321d4bfbc4f248c79ac584554c00b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:36.995274 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2e197-d33b-4bed-9415-5836fc50b065 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:37.043444 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-29b97891-01b4-44ef-a259-38260a2453ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71e74ae793254faeafcae5fffff258e9', 'request_id': 'req-29b97891-01b4-44ef-a259-38260a2453ec'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:37.057189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-29b97891-01b4-44ef-a259-38260a2453ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71e74ae793254faeafcae5fffff258e9', 'request_id': 'req-29b97891-01b4-44ef-a259-38260a2453ec'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:37.068106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-29b97891-01b4-44ef-a259-38260a2453ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71e74ae793254faeafcae5fffff258e9', 'request_id': 'req-29b97891-01b4-44ef-a259-38260a2453ec'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:37.076038 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ffe2e197-d33b-4bed-9415-5836fc50b065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbf321d4bfbc4f248c79ac584554c00b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:37.092130 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-29b97891-01b4-44ef-a259-38260a2453ec admin admin] The token cache is being invalidate because project 71e74ae793254faeafcae5fffff258e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:37.092476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-29b97891-01b4-44ef-a259-38260a2453ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:37.093289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-29b97891-01b4-44ef-a259-38260a2453ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:37.095538 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ffe2e197-d33b-4bed-9415-5836fc50b065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbf321d4bfbc4f248c79ac584554c00b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:37.101787 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 996/2000] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:36 2022] DELETE /identity/v3/projects/71e74ae793254faeafcae5fffff258e9 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:37.101787 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b97891-01b4-44ef-a259-38260a2453ec admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:37.101787 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-29b97891-01b4-44ef-a259-38260a2453ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:37.133143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ffe2e197-d33b-4bed-9415-5836fc50b065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbf321d4bfbc4f248c79ac584554c00b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:37.134390 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ffe2e197-d33b-4bed-9415-5836fc50b065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbf321d4bfbc4f248c79ac584554c00b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:37.153522 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ffe2e197-d33b-4bed-9415-5836fc50b065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbf321d4bfbc4f248c79ac584554c00b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:37.170794 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1005/2001] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:36 2022] DELETE /identity/v3/users/fbf321d4bfbc4f248c79ac584554c00b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:37.170794 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2e197-d33b-4bed-9415-5836fc50b065 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:37.170794 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ffe2e197-d33b-4bed-9415-5836fc50b065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:37.202112 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b97891-01b4-44ef-a259-38260a2453ec admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:37.289856 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2e197-d33b-4bed-9415-5836fc50b065 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:37.290119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:37.298507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8a3b9-5d98-4596-b13c-9c607a7309d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:37.299292 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8a3b9-5d98-4596-b13c-9c607a7309d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:37.299777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8a3b9-5d98-4596-b13c-9c607a7309d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:37.300251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8a3b9-5d98-4596-b13c-9c607a7309d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:37.301413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8a3b9-5d98-4596-b13c-9c607a7309d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:37.303856 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8a3b9-5d98-4596-b13c-9c607a7309d0 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:37.305073 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835354e-cfe1-420d-ba80-2e0f420ceb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:37.306074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835354e-cfe1-420d-ba80-2e0f420ceb4c admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:37.306999 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835354e-cfe1-420d-ba80-2e0f420ceb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:37.307996 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835354e-cfe1-420d-ba80-2e0f420ceb4c admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:37.308970 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835354e-cfe1-420d-ba80-2e0f420ceb4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:37.311455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835354e-cfe1-420d-ba80-2e0f420ceb4c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:37.319595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1006/2002] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:37.326099 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e9adc-3d95-42a2-aa6e-34cd7c825ecc None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:37.326598 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e9adc-3d95-42a2-aa6e-34cd7c825ecc None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:37.326964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e9adc-3d95-42a2-aa6e-34cd7c825ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:37.348505 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 997/2003] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:37 2022] POST /identity/v3/users => generated 287 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:37.353308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00e065-b3bf-4057-9c52-8ae8ab0c70f9 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:37.353308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00e065-b3bf-4057-9c52-8ae8ab0c70f9 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:37.353308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00e065-b3bf-4057-9c52-8ae8ab0c70f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:37.361679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-9a2e9adc-3d95-42a2-aa6e-34cd7c825ecc None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:37.363697 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9a2e9adc-3d95-42a2-aa6e-34cd7c825ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:37.376508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1007/2004] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:37.378011 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e9adc-3d95-42a2-aa6e-34cd7c825ecc None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:37.388094 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1d00e065-b3bf-4057-9c52-8ae8ab0c70f9 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:37.389647 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1d00e065-b3bf-4057-9c52-8ae8ab0c70f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:37.395183 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e9adc-3d95-42a2-aa6e-34cd7c825ecc None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:37.400359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 998/2005] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:37.405623 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00e065-b3bf-4057-9c52-8ae8ab0c70f9 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:37.405797 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06a40b-fe11-4822-958d-8264a6d6116b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:37.405990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1d00e065-b3bf-4057-9c52-8ae8ab0c70f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:37.406147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06a40b-fe11-4822-958d-8264a6d6116b admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:37.406147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06a40b-fe11-4822-958d-8264a6d6116b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:37.406147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06a40b-fe11-4822-958d-8264a6d6116b admin admin] PATH_INFO: `/v3/projects/06f2dd11c91840fab8fb40f9ef62783a/users/1f318dbedcdb465492e6435d8cb158be/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:37.420501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00e065-b3bf-4057-9c52-8ae8ab0c70f9 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:37.428176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3d71e-a9d4-4311-82f8-ec1177e12b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:37.428960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3d71e-a9d4-4311-82f8-ec1177e12b60 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:37.429258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3d71e-a9d4-4311-82f8-ec1177e12b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:37.429721 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3d71e-a9d4-4311-82f8-ec1177e12b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:37.430359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3d71e-a9d4-4311-82f8-ec1177e12b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:37.432564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3d71e-a9d4-4311-82f8-ec1177e12b60 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:37.442858 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06a40b-fe11-4822-958d-8264a6d6116b admin admin] RBAC: Authorizing `identity:create_grant(project_id=06f2dd11c91840fab8fb40f9ef62783a, user_id=1f318dbedcdb465492e6435d8cb158be, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:37.455974 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06a40b-fe11-4822-958d-8264a6d6116b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:37.466392 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 999/2006] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:37.470599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7719b44-fcaf-4d5d-85e1-5b1b4af23abb None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:37.471853 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7719b44-fcaf-4d5d-85e1-5b1b4af23abb None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:37.472231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7719b44-fcaf-4d5d-85e1-5b1b4af23abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:37.505192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1008/2007] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:37 2022] PUT /identity/v3/projects/06f2dd11c91840fab8fb40f9ef62783a/users/1f318dbedcdb465492e6435d8cb158be/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:37.508202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06a40b-fe11-4822-958d-8264a6d6116b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:37.509209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b7719b44-fcaf-4d5d-85e1-5b1b4af23abb None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:37.510828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b7719b44-fcaf-4d5d-85e1-5b1b4af23abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:37.512827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-dd06a40b-fe11-4822-958d-8264a6d6116b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:37.532114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:37.601153 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1000/2008] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:37.606908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7719b44-fcaf-4d5d-85e1-5b1b4af23abb None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:37.638466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7719b44-fcaf-4d5d-85e1-5b1b4af23abb None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:37.647974 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325db5b-eabf-4cf7-b18e-8201a16a017c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:37.648878 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325db5b-eabf-4cf7-b18e-8201a16a017c admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:37.650706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325db5b-eabf-4cf7-b18e-8201a16a017c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:37.651151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325db5b-eabf-4cf7-b18e-8201a16a017c admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:37.655639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325db5b-eabf-4cf7-b18e-8201a16a017c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:37.658406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325db5b-eabf-4cf7-b18e-8201a16a017c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:37.681979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1001/2009] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:37.687099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbe9ca-86c8-4969-b089-ea77ae057d29 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:37.687099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbe9ca-86c8-4969-b089-ea77ae057d29 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:37.687099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbe9ca-86c8-4969-b089-ea77ae057d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:37.750005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d7cbe9ca-86c8-4969-b089-ea77ae057d29 None None] MFA Rules not processed for user `1f318dbedcdb465492e6435d8cb158be`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:37.751613 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d7cbe9ca-86c8-4969-b089-ea77ae057d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:37.797545 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06a40b-fe11-4822-958d-8264a6d6116b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:37.805378 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97b79f-d82e-4cf7-9dba-fe11bc2d94e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:37.806230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97b79f-d82e-4cf7-9dba-fe11bc2d94e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:37.807274 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97b79f-d82e-4cf7-9dba-fe11bc2d94e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:37.809387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97b79f-d82e-4cf7-9dba-fe11bc2d94e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:37.848230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97b79f-d82e-4cf7-9dba-fe11bc2d94e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:37.853059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97b79f-d82e-4cf7-9dba-fe11bc2d94e2 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:37.896869 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1009/2010] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:37.915695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97b79f-d82e-4cf7-9dba-fe11bc2d94e2 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:37.958324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97b79f-d82e-4cf7-9dba-fe11bc2d94e2 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:37.977160 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333bf143-3026-4c2b-bf77-54cf67c9ef93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:37.979938 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333bf143-3026-4c2b-bf77-54cf67c9ef93 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:37.980505 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333bf143-3026-4c2b-bf77-54cf67c9ef93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:37.980998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333bf143-3026-4c2b-bf77-54cf67c9ef93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:37.982497 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333bf143-3026-4c2b-bf77-54cf67c9ef93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:37.983093 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333bf143-3026-4c2b-bf77-54cf67c9ef93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:37.989721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333bf143-3026-4c2b-bf77-54cf67c9ef93 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:38.019628 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1002/2011] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:38.021080 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cbe9ca-86c8-4969-b089-ea77ae057d29 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:38.048725 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1010/2012] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:38.048725 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333bf143-3026-4c2b-bf77-54cf67c9ef93 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:38.050582 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cbe9ca-86c8-4969-b089-ea77ae057d29 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:38.058613 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64e0dd-8883-49d5-ab3b-fa4877cbeb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:38.059439 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64e0dd-8883-49d5-ab3b-fa4877cbeb73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:38.059682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64e0dd-8883-49d5-ab3b-fa4877cbeb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:38.059960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64e0dd-8883-49d5-ab3b-fa4877cbeb73 admin admin] PATH_INFO: `/v3/projects/7778fdf4a6304f129177bff854f2922c/users/5b8ce07e8e254345a7487be7869cf82c/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:38.072659 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333bf143-3026-4c2b-bf77-54cf67c9ef93 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:38.083051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339a5d7-ac52-487a-9b2f-8d5a104f6ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:38.084486 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339a5d7-ac52-487a-9b2f-8d5a104f6ed8 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:38.085207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339a5d7-ac52-487a-9b2f-8d5a104f6ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:38.085897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339a5d7-ac52-487a-9b2f-8d5a104f6ed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:38.088760 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339a5d7-ac52-487a-9b2f-8d5a104f6ed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:38.094426 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64e0dd-8883-49d5-ab3b-fa4877cbeb73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7778fdf4a6304f129177bff854f2922c, user_id=5b8ce07e8e254345a7487be7869cf82c, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:38.095138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339a5d7-ac52-487a-9b2f-8d5a104f6ed8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:38.101526 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64e0dd-8883-49d5-ab3b-fa4877cbeb73 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:38.133198 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1003/2013] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:38 2022] PUT /identity/v3/projects/7778fdf4a6304f129177bff854f2922c/users/5b8ce07e8e254345a7487be7869cf82c/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:38.143705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64e0dd-8883-49d5-ab3b-fa4877cbeb73 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:38.186792 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1011/2014] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:38 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:38.190275 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1144d88-28f7-4cdf-a867-6ea9c6bd435a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:38.190645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1144d88-28f7-4cdf-a867-6ea9c6bd435a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:38.190962 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1144d88-28f7-4cdf-a867-6ea9c6bd435a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:38.210228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:38.254300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f1144d88-28f7-4cdf-a867-6ea9c6bd435a None None] MFA Rules not processed for user `5b8ce07e8e254345a7487be7869cf82c`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:38.257920 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f1144d88-28f7-4cdf-a867-6ea9c6bd435a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:38.296962 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64e0dd-8883-49d5-ab3b-fa4877cbeb73 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:38.387194 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5df68-3561-4357-8d3f-9699b4e1fa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:38.395820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5df68-3561-4357-8d3f-9699b4e1fa80 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:38.395820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5df68-3561-4357-8d3f-9699b4e1fa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:38.395820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5df68-3561-4357-8d3f-9699b4e1fa80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:38.395820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5df68-3561-4357-8d3f-9699b4e1fa80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:38.395820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5df68-3561-4357-8d3f-9699b4e1fa80 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:38.450265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1004/2015] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:38 2022] POST /identity/v3/projects => generated 377 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:38.451279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5df68-3561-4357-8d3f-9699b4e1fa80 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:38.503702 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1012/2016] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:38.503702 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1144d88-28f7-4cdf-a867-6ea9c6bd435a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:38.522536 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1144d88-28f7-4cdf-a867-6ea9c6bd435a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:38.555521 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5df68-3561-4357-8d3f-9699b4e1fa80 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:38.564650 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95233c-8896-40bb-b028-7f84ddfc07cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:38.565301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95233c-8896-40bb-b028-7f84ddfc07cd admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:38.565585 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95233c-8896-40bb-b028-7f84ddfc07cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:38.565860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95233c-8896-40bb-b028-7f84ddfc07cd admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:38.566520 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95233c-8896-40bb-b028-7f84ddfc07cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:38.572232 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95233c-8896-40bb-b028-7f84ddfc07cd admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:38.604099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1005/2017] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:38 2022] POST /identity/v3/users => generated 297 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:38.605105 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95233c-8896-40bb-b028-7f84ddfc07cd admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:38.609830 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6f95233c-8896-40bb-b028-7f84ddfc07cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:38.650694 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259e1e1-311e-4a40-ae05-2950212bcc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:38.652356 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259e1e1-311e-4a40-ae05-2950212bcc13 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:38.653079 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259e1e1-311e-4a40-ae05-2950212bcc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:38.653608 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259e1e1-311e-4a40-ae05-2950212bcc13 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:38.654959 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259e1e1-311e-4a40-ae05-2950212bcc13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:38.659911 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259e1e1-311e-4a40-ae05-2950212bcc13 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:38.674918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:38.688296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:38.713010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95233c-8896-40bb-b028-7f84ddfc07cd admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:38.714184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1013/2018] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:38 2022] POST /identity/v3/users => generated 289 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:38.717277 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259e1e1-311e-4a40-ae05-2950212bcc13 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:38.717893 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e7375-5bb0-4cc0-8cbe-d7c965e3cea4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:38.718624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e7375-5bb0-4cc0-8cbe-d7c965e3cea4 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:38.718959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e7375-5bb0-4cc0-8cbe-d7c965e3cea4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:38.719236 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e7375-5bb0-4cc0-8cbe-d7c965e3cea4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:38.742760 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259e1e1-311e-4a40-ae05-2950212bcc13 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:38.752670 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a54eb-9190-4874-a63a-c0c757547b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:38.754892 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a54eb-9190-4874-a63a-c0c757547b1b admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:38.755858 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a54eb-9190-4874-a63a-c0c757547b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:38.756659 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a54eb-9190-4874-a63a-c0c757547b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:38.758190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a54eb-9190-4874-a63a-c0c757547b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:38.760307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a54eb-9190-4874-a63a-c0c757547b1b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:38.784646 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1014/2019] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:38.788113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a54eb-9190-4874-a63a-c0c757547b1b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:38.801987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e7375-5bb0-4cc0-8cbe-d7c965e3cea4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:38.803951 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e7375-5bb0-4cc0-8cbe-d7c965e3cea4 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:38.816325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1006/2020] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:38.817464 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e7375-5bb0-4cc0-8cbe-d7c965e3cea4 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:38.836676 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a54eb-9190-4874-a63a-c0c757547b1b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:38.840122 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e7375-5bb0-4cc0-8cbe-d7c965e3cea4 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:38.847916 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b929c-0b0b-491e-a2ef-02814e2ba793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:38.848546 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b929c-0b0b-491e-a2ef-02814e2ba793 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:38.849547 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b929c-0b0b-491e-a2ef-02814e2ba793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:38.849852 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b929c-0b0b-491e-a2ef-02814e2ba793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:38.852840 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f00a03-1c6d-4c8e-bbe3-94519b10eba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:38.853106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f00a03-1c6d-4c8e-bbe3-94519b10eba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:38.853282 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f00a03-1c6d-4c8e-bbe3-94519b10eba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:38.853475 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f00a03-1c6d-4c8e-bbe3-94519b10eba7 admin admin] PATH_INFO: `/v3/projects/4f48e578ec97412a9eb3666595c4743f/users/aa8f0e09338e46d787ae61e8fe03d546/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:38.854302 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b929c-0b0b-491e-a2ef-02814e2ba793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:38.856475 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b929c-0b0b-491e-a2ef-02814e2ba793 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:38.886868 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1015/2021] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:38.890357 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b929c-0b0b-491e-a2ef-02814e2ba793 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:38.896266 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f00a03-1c6d-4c8e-bbe3-94519b10eba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f48e578ec97412a9eb3666595c4743f, user_id=aa8f0e09338e46d787ae61e8fe03d546, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:38.899210 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f00a03-1c6d-4c8e-bbe3-94519b10eba7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:38.913333 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b929c-0b0b-491e-a2ef-02814e2ba793 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:38.920464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f71a5-903c-4b41-b658-25b218a33fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:38.921115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f71a5-903c-4b41-b658-25b218a33fec admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:38.921504 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f71a5-903c-4b41-b658-25b218a33fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:38.921886 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f71a5-903c-4b41-b658-25b218a33fec admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:38.922731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f71a5-903c-4b41-b658-25b218a33fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:38.939446 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1007/2022] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:38 2022] PUT /identity/v3/projects/4f48e578ec97412a9eb3666595c4743f/users/aa8f0e09338e46d787ae61e8fe03d546/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:38.939747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f71a5-903c-4b41-b658-25b218a33fec admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:38.941111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f00a03-1c6d-4c8e-bbe3-94519b10eba7 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:38.971311 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1016/2023] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:38.972935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f71a5-903c-4b41-b658-25b218a33fec admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:38.997244 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f71a5-903c-4b41-b658-25b218a33fec admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:39.131297 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f00a03-1c6d-4c8e-bbe3-94519b10eba7 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:39.140282 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0133dc-7b4f-4dcb-a11d-4f576cf8e92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:39.141264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0133dc-7b4f-4dcb-a11d-4f576cf8e92b admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:39.141635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310603c1-3cf0-4450-8232-0f0e393ef95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:39.141771 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0133dc-7b4f-4dcb-a11d-4f576cf8e92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:39.142211 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0133dc-7b4f-4dcb-a11d-4f576cf8e92b admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:39.142799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310603c1-3cf0-4450-8232-0f0e393ef95b admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:39.143112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310603c1-3cf0-4450-8232-0f0e393ef95b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:39.143355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0133dc-7b4f-4dcb-a11d-4f576cf8e92b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:39.143464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310603c1-3cf0-4450-8232-0f0e393ef95b admin admin] PATH_INFO: `/v3/projects/754375a274c54a84a7db99c33218c97a/users/96c766aa092c4af185687decfb0db0f0/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:39.146770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0133dc-7b4f-4dcb-a11d-4f576cf8e92b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:39.191306 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1008/2024] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:39.198769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0133dc-7b4f-4dcb-a11d-4f576cf8e92b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:39.206332 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310603c1-3cf0-4450-8232-0f0e393ef95b admin admin] RBAC: Authorizing `identity:create_grant(project_id=754375a274c54a84a7db99c33218c97a, user_id=96c766aa092c4af185687decfb0db0f0, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:39.220653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310603c1-3cf0-4450-8232-0f0e393ef95b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:39.230252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0133dc-7b4f-4dcb-a11d-4f576cf8e92b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:39.237787 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51576958-55c9-4366-897c-b57e5328d349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:39.238412 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51576958-55c9-4366-897c-b57e5328d349 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:39.238724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51576958-55c9-4366-897c-b57e5328d349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:39.239005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51576958-55c9-4366-897c-b57e5328d349 admin admin] PATH_INFO: `/v3/projects/4f48e578ec97412a9eb3666595c4743f/users/aa8f0e09338e46d787ae61e8fe03d546/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:39.241829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51576958-55c9-4366-897c-b57e5328d349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f48e578ec97412a9eb3666595c4743f, user_id=aa8f0e09338e46d787ae61e8fe03d546, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:39.244831 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51576958-55c9-4366-897c-b57e5328d349 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:39.250884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1017/2025] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:38 2022] PUT /identity/v3/projects/754375a274c54a84a7db99c33218c97a/users/96c766aa092c4af185687decfb0db0f0/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:39.253777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310603c1-3cf0-4450-8232-0f0e393ef95b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:39.259669 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-310603c1-3cf0-4450-8232-0f0e393ef95b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:39.286696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1009/2026] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:39 2022] PUT /identity/v3/projects/4f48e578ec97412a9eb3666595c4743f/users/aa8f0e09338e46d787ae61e8fe03d546/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:39.286696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d7be4-cc5b-49ff-8271-ded20c9ff6f3 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:39.286696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d7be4-cc5b-49ff-8271-ded20c9ff6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:39.286696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d7be4-cc5b-49ff-8271-ded20c9ff6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:39.319399 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-2b5d7be4-cc5b-49ff-8271-ded20c9ff6f3 None None] MFA Rules not processed for user `96c766aa092c4af185687decfb0db0f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:39.320990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2b5d7be4-cc5b-49ff-8271-ded20c9ff6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:39.435491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:39.473365 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310603c1-3cf0-4450-8232-0f0e393ef95b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:39.476907 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1010/2027] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:39.478131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc4f33-471a-41d9-b99e-c2b5805e709d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:39.478424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc4f33-471a-41d9-b99e-c2b5805e709d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:39.478719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc4f33-471a-41d9-b99e-c2b5805e709d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:39.516705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-5dfc4f33-471a-41d9-b99e-c2b5805e709d None None] MFA Rules not processed for user `aa8f0e09338e46d787ae61e8fe03d546`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:39.518633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5dfc4f33-471a-41d9-b99e-c2b5805e709d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:39.591717 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:39.631224 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c2147-1675-4b16-817a-1c34bd0c0c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:39.632770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c2147-1675-4b16-817a-1c34bd0c0c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:39.633397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c2147-1675-4b16-817a-1c34bd0c0c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:39.633943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c2147-1675-4b16-817a-1c34bd0c0c95 admin admin] PATH_INFO: `/v3/projects/5b7b1d0918ee42b8b2d779f84874ea36` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:39.637145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c2147-1675-4b16-817a-1c34bd0c0c95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b7b1d0918ee42b8b2d779f84874ea36)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:39.641269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c2147-1675-4b16-817a-1c34bd0c0c95 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:39.673731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1011/2028] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:39.673731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc4f33-471a-41d9-b99e-c2b5805e709d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:39.681024 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc4f33-471a-41d9-b99e-c2b5805e709d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:39.687291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28508eef-71a1-4e66-a9fb-804ae0b60468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:39.688189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28508eef-71a1-4e66-a9fb-804ae0b60468 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:39.688460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28508eef-71a1-4e66-a9fb-804ae0b60468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:39.688875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28508eef-71a1-4e66-a9fb-804ae0b60468 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:39.689800 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28508eef-71a1-4e66-a9fb-804ae0b60468 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:39.691927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28508eef-71a1-4e66-a9fb-804ae0b60468 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:39.768100 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1012/2029] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:39 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:39.770260 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28508eef-71a1-4e66-a9fb-804ae0b60468 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:39.819516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6d4c2147-1675-4b16-817a-1c34bd0c0c95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b7b1d0918ee42b8b2d779f84874ea36', 'request_id': 'req-6d4c2147-1675-4b16-817a-1c34bd0c0c95'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:39.835416 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6d4c2147-1675-4b16-817a-1c34bd0c0c95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b7b1d0918ee42b8b2d779f84874ea36', 'request_id': 'req-6d4c2147-1675-4b16-817a-1c34bd0c0c95'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:39.849856 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6d4c2147-1675-4b16-817a-1c34bd0c0c95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b7b1d0918ee42b8b2d779f84874ea36', 'request_id': 'req-6d4c2147-1675-4b16-817a-1c34bd0c0c95'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:39.870840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6d4c2147-1675-4b16-817a-1c34bd0c0c95 admin admin] The token cache is being invalidate because project 5b7b1d0918ee42b8b2d779f84874ea36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:39.871360 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6d4c2147-1675-4b16-817a-1c34bd0c0c95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:39.872926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6d4c2147-1675-4b16-817a-1c34bd0c0c95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:39.875483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1018/2030] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:39 2022] DELETE /identity/v3/projects/5b7b1d0918ee42b8b2d779f84874ea36 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:39.879672 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c2147-1675-4b16-817a-1c34bd0c0c95 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:39.881400 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6d4c2147-1675-4b16-817a-1c34bd0c0c95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:39.893700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28508eef-71a1-4e66-a9fb-804ae0b60468 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:39.988638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c2147-1675-4b16-817a-1c34bd0c0c95 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:39.994593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bfdc2-76b9-468f-b42f-7cbbd8731c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:39.995360 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bfdc2-76b9-468f-b42f-7cbbd8731c08 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:39.995711 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bfdc2-76b9-468f-b42f-7cbbd8731c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:39.996086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bfdc2-76b9-468f-b42f-7cbbd8731c08 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:39.996903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ead91-8a63-4e74-b2e9-b89c681cc191 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:39.997065 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bfdc2-76b9-468f-b42f-7cbbd8731c08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:39.997825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ead91-8a63-4e74-b2e9-b89c681cc191 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:39.998452 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ead91-8a63-4e74-b2e9-b89c681cc191 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:39.999121 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ead91-8a63-4e74-b2e9-b89c681cc191 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:39.999971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bfdc2-76b9-468f-b42f-7cbbd8731c08 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:40.002562 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-224ead91-8a63-4e74-b2e9-b89c681cc191 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:40.030849 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1019/2031] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:39 2022] POST /identity/v3/users => generated 297 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:40.030849 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bfdc2-76b9-468f-b42f-7cbbd8731c08 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:40.031728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d64bfdc2-76b9-468f-b42f-7cbbd8731c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:40.042678 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:40.047419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bfdc2-76b9-468f-b42f-7cbbd8731c08 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:40.053730 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd4506-3adc-4730-aea1-abfd3e65652a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:40.054486 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd4506-3adc-4730-aea1-abfd3e65652a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:40.055151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd4506-3adc-4730-aea1-abfd3e65652a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:40.055289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd4506-3adc-4730-aea1-abfd3e65652a admin admin] PATH_INFO: `/v3/users/0df6253b6d9243f6a255fd364f2da422` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:40.056943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd4506-3adc-4730-aea1-abfd3e65652a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0df6253b6d9243f6a255fd364f2da422)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:40.059791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd4506-3adc-4730-aea1-abfd3e65652a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:40.144561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-89dd4506-3adc-4730-aea1-abfd3e65652a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0df6253b6d9243f6a255fd364f2da422'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:40.149493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ead91-8a63-4e74-b2e9-b89c681cc191 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:40.152580 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ead91-8a63-4e74-b2e9-b89c681cc191 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:40.160137 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-89dd4506-3adc-4730-aea1-abfd3e65652a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0df6253b6d9243f6a255fd364f2da422'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:40.179099 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:40.185639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1013/2032] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:40.186256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ead91-8a63-4e74-b2e9-b89c681cc191 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:40.202902 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-89dd4506-3adc-4730-aea1-abfd3e65652a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0df6253b6d9243f6a255fd364f2da422'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:40.204069 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-89dd4506-3adc-4730-aea1-abfd3e65652a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0df6253b6d9243f6a255fd364f2da422'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:40.210626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ead91-8a63-4e74-b2e9-b89c681cc191 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:40.217124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-89dd4506-3adc-4730-aea1-abfd3e65652a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0df6253b6d9243f6a255fd364f2da422'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:40.220949 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec242d-b9fe-438d-8172-a4504678484c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:40.221529 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec242d-b9fe-438d-8172-a4504678484c admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:40.222001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec242d-b9fe-438d-8172-a4504678484c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:40.222489 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec242d-b9fe-438d-8172-a4504678484c admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:40.223719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec242d-b9fe-438d-8172-a4504678484c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:40.226245 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1020/2033] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:40 2022] DELETE /identity/v3/users/0df6253b6d9243f6a255fd364f2da422 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:40.226718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec242d-b9fe-438d-8172-a4504678484c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:40.246101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1014/2034] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:40.251911 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd4506-3adc-4730-aea1-abfd3e65652a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:40.254298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-89dd4506-3adc-4730-aea1-abfd3e65652a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:40.350424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec242d-b9fe-438d-8172-a4504678484c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:40.352339 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-37ec242d-b9fe-438d-8172-a4504678484c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:40.383108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec242d-b9fe-438d-8172-a4504678484c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:40.412935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd4506-3adc-4730-aea1-abfd3e65652a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:40.420396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30d99e-9976-425b-a73a-c2776bc3bf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:40.421099 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30d99e-9976-425b-a73a-c2776bc3bf6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:40.421382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30d99e-9976-425b-a73a-c2776bc3bf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:40.421659 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30d99e-9976-425b-a73a-c2776bc3bf6f admin admin] PATH_INFO: `/v3/projects/5afa858b8dfa4e9095e28736b579b4ef/users/7da0358d74db43fbb541c80ffe9b8f8f/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:40.445464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30d99e-9976-425b-a73a-c2776bc3bf6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afa858b8dfa4e9095e28736b579b4ef, user_id=7da0358d74db43fbb541c80ffe9b8f8f, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:40.448625 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30d99e-9976-425b-a73a-c2776bc3bf6f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:40.457456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f45282-9ca1-4c07-a551-997033679448 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:40.458520 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f45282-9ca1-4c07-a551-997033679448 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:40.458520 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f45282-9ca1-4c07-a551-997033679448 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:40.458912 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f45282-9ca1-4c07-a551-997033679448 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:40.465671 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b2f45282-9ca1-4c07-a551-997033679448 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:40.476093 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1021/2035] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:40 2022] PUT /identity/v3/projects/5afa858b8dfa4e9095e28736b579b4ef/users/7da0358d74db43fbb541c80ffe9b8f8f/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:40.477307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30d99e-9976-425b-a73a-c2776bc3bf6f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:40.535768 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f45282-9ca1-4c07-a551-997033679448 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:40.537550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f45282-9ca1-4c07-a551-997033679448 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:40.550811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1015/2036] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:15:40.590732 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30d99e-9976-425b-a73a-c2776bc3bf6f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:40.600980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96579dea-f90a-40e8-b8bd-758950fd563c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:40.607198 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96579dea-f90a-40e8-b8bd-758950fd563c admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:40.607645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96579dea-f90a-40e8-b8bd-758950fd563c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:40.610226 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96579dea-f90a-40e8-b8bd-758950fd563c admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:40.615612 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96579dea-f90a-40e8-b8bd-758950fd563c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:40.617632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96579dea-f90a-40e8-b8bd-758950fd563c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:40.633363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1022/2037] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:40.639126 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f45282-9ca1-4c07-a551-997033679448 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:40.656353 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f45282-9ca1-4c07-a551-997033679448 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:40.673145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dcfca2-8c3e-482e-9033-293364c58507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:40.674825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcfca2-8c3e-482e-9033-293364c58507 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:40.675487 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcfca2-8c3e-482e-9033-293364c58507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:40.676560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcfca2-8c3e-482e-9033-293364c58507 admin admin] PATH_INFO: `/v3/projects/5afa858b8dfa4e9095e28736b579b4ef/users/7da0358d74db43fbb541c80ffe9b8f8f/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:40.683080 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dcfca2-8c3e-482e-9033-293364c58507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afa858b8dfa4e9095e28736b579b4ef, user_id=7da0358d74db43fbb541c80ffe9b8f8f, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:40.700109 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dcfca2-8c3e-482e-9033-293364c58507 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:40.743527 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1016/2038] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:40 2022] PUT /identity/v3/projects/5afa858b8dfa4e9095e28736b579b4ef/users/7da0358d74db43fbb541c80ffe9b8f8f/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:40.756650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101d76b-2a0a-4e10-9a0a-c143e85e9291 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:40.756650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101d76b-2a0a-4e10-9a0a-c143e85e9291 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:40.756650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101d76b-2a0a-4e10-9a0a-c143e85e9291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:40.771905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dcfca2-8c3e-482e-9033-293364c58507 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:40.774713 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a3dcfca2-8c3e-482e-9033-293364c58507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:40.862279 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-0101d76b-2a0a-4e10-9a0a-c143e85e9291 None None] MFA Rules not processed for user `7da0358d74db43fbb541c80ffe9b8f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:40.870108 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0101d76b-2a0a-4e10-9a0a-c143e85e9291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:40.944930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:40.992552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dcfca2-8c3e-482e-9033-293364c58507 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:41.011459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25edabc-fa82-445a-b24d-820daf079fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:41.012122 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25edabc-fa82-445a-b24d-820daf079fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.012401 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25edabc-fa82-445a-b24d-820daf079fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.012682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25edabc-fa82-445a-b24d-820daf079fde admin admin] PATH_INFO: `/v3/projects/44b8ef0581aa49d38f3ad7e6d3d3e86d` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.019635 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25edabc-fa82-445a-b24d-820daf079fde admin admin] RBAC: Authorizing `identity:delete_project(project_id=44b8ef0581aa49d38f3ad7e6d3d3e86d)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:41.023791 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25edabc-fa82-445a-b24d-820daf079fde admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:41.027796 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:41.115823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1023/2039] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:15:41.125052 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101d76b-2a0a-4e10-9a0a-c143e85e9291 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:41.127563 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0101d76b-2a0a-4e10-9a0a-c143e85e9291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:41.162235 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101d76b-2a0a-4e10-9a0a-c143e85e9291 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:41.248256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c25edabc-fa82-445a-b24d-820daf079fde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44b8ef0581aa49d38f3ad7e6d3d3e86d', 'request_id': 'req-c25edabc-fa82-445a-b24d-820daf079fde'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:41.254184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acffe79-ca5e-4a6b-a5f6-524eaed550bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:41.255120 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acffe79-ca5e-4a6b-a5f6-524eaed550bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.255472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acffe79-ca5e-4a6b-a5f6-524eaed550bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.256451 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acffe79-ca5e-4a6b-a5f6-524eaed550bb admin admin] PATH_INFO: `/v3/users/1f318dbedcdb465492e6435d8cb158be` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.257227 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acffe79-ca5e-4a6b-a5f6-524eaed550bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f318dbedcdb465492e6435d8cb158be)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:41.259709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acffe79-ca5e-4a6b-a5f6-524eaed550bb admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:41.272168 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c25edabc-fa82-445a-b24d-820daf079fde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44b8ef0581aa49d38f3ad7e6d3d3e86d', 'request_id': 'req-c25edabc-fa82-445a-b24d-820daf079fde'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:41.285001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c25edabc-fa82-445a-b24d-820daf079fde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44b8ef0581aa49d38f3ad7e6d3d3e86d', 'request_id': 'req-c25edabc-fa82-445a-b24d-820daf079fde'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:41.321440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c25edabc-fa82-445a-b24d-820daf079fde admin admin] The token cache is being invalidate because project 44b8ef0581aa49d38f3ad7e6d3d3e86d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:41.321913 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c25edabc-fa82-445a-b24d-820daf079fde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:41.323149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c25edabc-fa82-445a-b24d-820daf079fde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:41.327996 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1017/2040] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:40 2022] DELETE /identity/v3/projects/44b8ef0581aa49d38f3ad7e6d3d3e86d => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:41.328525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25edabc-fa82-445a-b24d-820daf079fde admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:41.335115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c25edabc-fa82-445a-b24d-820daf079fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:41.362651 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4acffe79-ca5e-4a6b-a5f6-524eaed550bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f318dbedcdb465492e6435d8cb158be'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:41.400382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4acffe79-ca5e-4a6b-a5f6-524eaed550bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f318dbedcdb465492e6435d8cb158be'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:41.433580 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4acffe79-ca5e-4a6b-a5f6-524eaed550bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f318dbedcdb465492e6435d8cb158be'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:41.435214 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4acffe79-ca5e-4a6b-a5f6-524eaed550bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f318dbedcdb465492e6435d8cb158be'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:41.464281 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4acffe79-ca5e-4a6b-a5f6-524eaed550bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f318dbedcdb465492e6435d8cb158be'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:41.483075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1024/2041] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:41 2022] DELETE /identity/v3/users/1f318dbedcdb465492e6435d8cb158be => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:41.483075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f5b18-2ba2-4aad-8e6d-87100d3a7423 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.483075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f5b18-2ba2-4aad-8e6d-87100d3a7423 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.483631 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f5b18-2ba2-4aad-8e6d-87100d3a7423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.532234 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25edabc-fa82-445a-b24d-820daf079fde admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:41.538252 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-125f5b18-2ba2-4aad-8e6d-87100d3a7423 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:41.541751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-125f5b18-2ba2-4aad-8e6d-87100d3a7423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:41.663194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1025/2042] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:41.666092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af54479-dc39-47eb-93fb-331014d0b754 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.668417 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd3492-47f1-4afb-908a-1362103e771c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:41.670156 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd3492-47f1-4afb-908a-1362103e771c service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.670692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd3492-47f1-4afb-908a-1362103e771c service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.671146 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af54479-dc39-47eb-93fb-331014d0b754 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.672254 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af54479-dc39-47eb-93fb-331014d0b754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.675459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd3492-47f1-4afb-908a-1362103e771c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.681120 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f8cd3492-47f1-4afb-908a-1362103e771c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:41.693573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:41.729225 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-1af54479-dc39-47eb-93fb-331014d0b754 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:41.742708 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1af54479-dc39-47eb-93fb-331014d0b754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:41.757468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1026/2043] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:41.768906 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866715ca-5831-4266-9853-c01f2770f5ef None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.771557 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866715ca-5831-4266-9853-c01f2770f5ef None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.772017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866715ca-5831-4266-9853-c01f2770f5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:41.797097 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-866715ca-5831-4266-9853-c01f2770f5ef None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:41.799452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-866715ca-5831-4266-9853-c01f2770f5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:41.818138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1027/2044] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:41.819364 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866715ca-5831-4266-9853-c01f2770f5ef None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:41.820612 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:41.821276 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-866715ca-5831-4266-9853-c01f2770f5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:41.904810 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd3492-47f1-4afb-908a-1362103e771c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:41.909633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd3492-47f1-4afb-908a-1362103e771c service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:41.949119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1018/2045] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:41.959286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd3492-47f1-4afb-908a-1362103e771c service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:41.968501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866715ca-5831-4266-9853-c01f2770f5ef None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:41.982214 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85319c0-002b-4442-aeac-f2f36fd6ebde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:41.983391 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85319c0-002b-4442-aeac-f2f36fd6ebde service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:41.983958 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85319c0-002b-4442-aeac-f2f36fd6ebde service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:41.984520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85319c0-002b-4442-aeac-f2f36fd6ebde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.007594 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd3492-47f1-4afb-908a-1362103e771c service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.024695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85319c0-002b-4442-aeac-f2f36fd6ebde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.027609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85319c0-002b-4442-aeac-f2f36fd6ebde service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.031257 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad52e43-161e-4f3e-803d-1d8776c9d616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.032320 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad52e43-161e-4f3e-803d-1d8776c9d616 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.033333 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad52e43-161e-4f3e-803d-1d8776c9d616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.034876 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad52e43-161e-4f3e-803d-1d8776c9d616 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.040379 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad52e43-161e-4f3e-803d-1d8776c9d616 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:42.040928 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad52e43-161e-4f3e-803d-1d8776c9d616 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.049783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad52e43-161e-4f3e-803d-1d8776c9d616 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.082718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1019/2046] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:42.085683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad52e43-161e-4f3e-803d-1d8776c9d616 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.091308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1028/2047] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:42.101501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad52e43-161e-4f3e-803d-1d8776c9d616 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.108909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c104a2-618a-42d8-a021-c2688e0c4e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.109737 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c104a2-618a-42d8-a021-c2688e0c4e49 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.110179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c104a2-618a-42d8-a021-c2688e0c4e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.111261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c104a2-618a-42d8-a021-c2688e0c4e49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.112466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c104a2-618a-42d8-a021-c2688e0c4e49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.116844 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c104a2-618a-42d8-a021-c2688e0c4e49 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.175723 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1020/2048] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/projects => generated 365 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:15:42.183813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85319c0-002b-4442-aeac-f2f36fd6ebde service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.199573 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c104a2-618a-42d8-a021-c2688e0c4e49 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.210976 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-58c104a2-618a-42d8-a021-c2688e0c4e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:42.239072 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85319c0-002b-4442-aeac-f2f36fd6ebde service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.403414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c104a2-618a-42d8-a021-c2688e0c4e49 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.416121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb9242-7c3b-4576-9443-779f66ffd67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.416761 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb9242-7c3b-4576-9443-779f66ffd67a admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.417043 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb9242-7c3b-4576-9443-779f66ffd67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.417314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb9242-7c3b-4576-9443-779f66ffd67a admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.420702 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb9242-7c3b-4576-9443-779f66ffd67a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.425080 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d80d5a-d457-4f8d-813f-d9eb0374a9f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.425735 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d80d5a-d457-4f8d-813f-d9eb0374a9f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.427046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb9242-7c3b-4576-9443-779f66ffd67a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.428993 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d80d5a-d457-4f8d-813f-d9eb0374a9f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.429268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d80d5a-d457-4f8d-813f-d9eb0374a9f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.444248 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d2d80d5a-d457-4f8d-813f-d9eb0374a9f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:42.537560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1029/2049] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/users => generated 283 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:42.543395 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb9242-7c3b-4576-9443-779f66ffd67a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.568850 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb9242-7c3b-4576-9443-779f66ffd67a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.582967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abdac8-900c-44ab-9de1-f3f1ca8501e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.583743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abdac8-900c-44ab-9de1-f3f1ca8501e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.584745 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abdac8-900c-44ab-9de1-f3f1ca8501e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.585155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abdac8-900c-44ab-9de1-f3f1ca8501e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.586900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23abdac8-900c-44ab-9de1-f3f1ca8501e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.596924 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23abdac8-900c-44ab-9de1-f3f1ca8501e6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.630363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1030/2050] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:42.631493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abdac8-900c-44ab-9de1-f3f1ca8501e6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.659624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abdac8-900c-44ab-9de1-f3f1ca8501e6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.691897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9f60d-c763-484e-9d16-9f1383259906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.693021 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9f60d-c763-484e-9d16-9f1383259906 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.694474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9f60d-c763-484e-9d16-9f1383259906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.695136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9f60d-c763-484e-9d16-9f1383259906 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.698378 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9f60d-c763-484e-9d16-9f1383259906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.703803 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d80d5a-d457-4f8d-813f-d9eb0374a9f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.706304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9f60d-c763-484e-9d16-9f1383259906 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.706754 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d80d5a-d457-4f8d-813f-d9eb0374a9f4 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.718818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1021/2051] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:42.764776 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1031/2052] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:42.769444 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d80d5a-d457-4f8d-813f-d9eb0374a9f4 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:42.798728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d80d5a-d457-4f8d-813f-d9eb0374a9f4 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:42.803667 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ed303-14d9-4e3e-8e1d-70bf85828ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:42.804346 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ed303-14d9-4e3e-8e1d-70bf85828ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.804625 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ed303-14d9-4e3e-8e1d-70bf85828ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.804897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ed303-14d9-4e3e-8e1d-70bf85828ff4 admin admin] PATH_INFO: `/v3/projects/15cf0fb91c77486488120f518735f991/users/76afe7068d604416a2621aee343f50b3/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.833342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ed303-14d9-4e3e-8e1d-70bf85828ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15cf0fb91c77486488120f518735f991, user_id=76afe7068d604416a2621aee343f50b3, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:42.837360 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ed303-14d9-4e3e-8e1d-70bf85828ff4 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:42.859622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1022/2053] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:42 2022] PUT /identity/v3/projects/15cf0fb91c77486488120f518735f991/users/76afe7068d604416a2621aee343f50b3/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:42.875391 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6bc01-0b02-4048-a007-d31f053b1b69 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:42.875895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6bc01-0b02-4048-a007-d31f053b1b69 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:42.876445 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6bc01-0b02-4048-a007-d31f053b1b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:42.933663 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c3b6bc01-0b02-4048-a007-d31f053b1b69 None None] MFA Rules not processed for user `76afe7068d604416a2621aee343f50b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:42.936860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c3b6bc01-0b02-4048-a007-d31f053b1b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:43.108260 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:43.172133 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1032/2054] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:43.175576 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ed303-14d9-4e3e-8e1d-70bf85828ff4 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.204875 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6bc01-0b02-4048-a007-d31f053b1b69 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.207633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c3b6bc01-0b02-4048-a007-d31f053b1b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:43.245596 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:43.306323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ed303-14d9-4e3e-8e1d-70bf85828ff4 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.313211 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57fbf3-92b3-4b27-9e21-cbce4e4923ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.314956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57fbf3-92b3-4b27-9e21-cbce4e4923ef admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.315228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57fbf3-92b3-4b27-9e21-cbce4e4923ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.315663 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57fbf3-92b3-4b27-9e21-cbce4e4923ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.316529 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57fbf3-92b3-4b27-9e21-cbce4e4923ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.318549 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57fbf3-92b3-4b27-9e21-cbce4e4923ef admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.361501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6bc01-0b02-4048-a007-d31f053b1b69 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.367940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1023/2055] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:43 2022] POST /identity/v3/projects => generated 365 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:43.369018 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411172f1-93a5-497b-a44b-b7c965d270cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.369761 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411172f1-93a5-497b-a44b-b7c965d270cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.370110 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411172f1-93a5-497b-a44b-b7c965d270cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.370452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411172f1-93a5-497b-a44b-b7c965d270cf admin admin] PATH_INFO: `/v3/projects/06f2dd11c91840fab8fb40f9ef62783a` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.371841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411172f1-93a5-497b-a44b-b7c965d270cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=06f2dd11c91840fab8fb40f9ef62783a)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.374018 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411172f1-93a5-497b-a44b-b7c965d270cf admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.375026 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57fbf3-92b3-4b27-9e21-cbce4e4923ef admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.468836 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-411172f1-93a5-497b-a44b-b7c965d270cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06f2dd11c91840fab8fb40f9ef62783a', 'request_id': 'req-411172f1-93a5-497b-a44b-b7c965d270cf'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:43.483289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-411172f1-93a5-497b-a44b-b7c965d270cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06f2dd11c91840fab8fb40f9ef62783a', 'request_id': 'req-411172f1-93a5-497b-a44b-b7c965d270cf'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:43.496912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-411172f1-93a5-497b-a44b-b7c965d270cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06f2dd11c91840fab8fb40f9ef62783a', 'request_id': 'req-411172f1-93a5-497b-a44b-b7c965d270cf'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:43.506254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57fbf3-92b3-4b27-9e21-cbce4e4923ef admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.546432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-411172f1-93a5-497b-a44b-b7c965d270cf admin admin] The token cache is being invalidate because project 06f2dd11c91840fab8fb40f9ef62783a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:43.547142 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-411172f1-93a5-497b-a44b-b7c965d270cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:43.549371 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-411172f1-93a5-497b-a44b-b7c965d270cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:43.552822 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1033/2056] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:43 2022] DELETE /identity/v3/projects/06f2dd11c91840fab8fb40f9ef62783a => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:43.559628 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411172f1-93a5-497b-a44b-b7c965d270cf admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.564182 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-411172f1-93a5-497b-a44b-b7c965d270cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:43.635606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d10302-6844-40de-b56b-a0d4f0443d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.636606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d10302-6844-40de-b56b-a0d4f0443d19 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.637257 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d10302-6844-40de-b56b-a0d4f0443d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.637740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d10302-6844-40de-b56b-a0d4f0443d19 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.639043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d10302-6844-40de-b56b-a0d4f0443d19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.643123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d10302-6844-40de-b56b-a0d4f0443d19 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.678702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1024/2057] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:43 2022] POST /identity/v3/users => generated 289 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:43.680075 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d10302-6844-40de-b56b-a0d4f0443d19 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.682464 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-85d10302-6844-40de-b56b-a0d4f0443d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:43.701793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d10302-6844-40de-b56b-a0d4f0443d19 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.709375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ec474-bc02-4800-8189-219f680b325f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.710150 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ec474-bc02-4800-8189-219f680b325f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.710150 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ec474-bc02-4800-8189-219f680b325f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.710444 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ec474-bc02-4800-8189-219f680b325f admin admin] PATH_INFO: `/v3/users/aa8f0e09338e46d787ae61e8fe03d546` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.711762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ec474-bc02-4800-8189-219f680b325f admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa8f0e09338e46d787ae61e8fe03d546)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.714263 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ec474-bc02-4800-8189-219f680b325f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.719782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411172f1-93a5-497b-a44b-b7c965d270cf admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:43.731100 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1eb7e-a377-400b-8882-cf3fff4b6460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:43.732404 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1eb7e-a377-400b-8882-cf3fff4b6460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.733524 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1eb7e-a377-400b-8882-cf3fff4b6460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.734641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1eb7e-a377-400b-8882-cf3fff4b6460 admin admin] PATH_INFO: `/v3/users/5b8ce07e8e254345a7487be7869cf82c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.739371 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1eb7e-a377-400b-8882-cf3fff4b6460 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b8ce07e8e254345a7487be7869cf82c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:43.741428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1eb7e-a377-400b-8882-cf3fff4b6460 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:43.808469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:43.824453 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8d1ec474-bc02-4800-8189-219f680b325f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa8f0e09338e46d787ae61e8fe03d546'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:43.834727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:43.841126 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8d1ec474-bc02-4800-8189-219f680b325f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa8f0e09338e46d787ae61e8fe03d546'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:43.861944 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-58c1eb7e-a377-400b-8882-cf3fff4b6460 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b8ce07e8e254345a7487be7869cf82c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:43.869092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8d1ec474-bc02-4800-8189-219f680b325f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa8f0e09338e46d787ae61e8fe03d546'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:43.870152 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8d1ec474-bc02-4800-8189-219f680b325f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa8f0e09338e46d787ae61e8fe03d546'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:43.870901 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8d1ec474-bc02-4800-8189-219f680b325f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa8f0e09338e46d787ae61e8fe03d546'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:43.879309 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-58c1eb7e-a377-400b-8882-cf3fff4b6460 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b8ce07e8e254345a7487be7869cf82c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:43.897393 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1025/2058] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:43 2022] DELETE /identity/v3/users/aa8f0e09338e46d787ae61e8fe03d546 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:43.899106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ec474-bc02-4800-8189-219f680b325f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:43.901786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8d1ec474-bc02-4800-8189-219f680b325f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:43.904299 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-58c1eb7e-a377-400b-8882-cf3fff4b6460 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b8ce07e8e254345a7487be7869cf82c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:43.905298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-58c1eb7e-a377-400b-8882-cf3fff4b6460 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b8ce07e8e254345a7487be7869cf82c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:43.918560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-58c1eb7e-a377-400b-8882-cf3fff4b6460 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b8ce07e8e254345a7487be7869cf82c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:43.932051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1034/2059] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:43 2022] DELETE /identity/v3/users/5b8ce07e8e254345a7487be7869cf82c => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:43.934478 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47e012-80b7-4e70-a920-8123325f82f0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:43.934910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47e012-80b7-4e70-a920-8123325f82f0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:43.935599 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47e012-80b7-4e70-a920-8123325f82f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:43.970867 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ed47e012-80b7-4e70-a920-8123325f82f0 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:43.974061 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ed47e012-80b7-4e70-a920-8123325f82f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:44.064815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ec474-bc02-4800-8189-219f680b325f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:44.149590 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645617bd-ce0b-4e37-9d77-9342f85226fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:44.150487 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645617bd-ce0b-4e37-9d77-9342f85226fc admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.150810 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645617bd-ce0b-4e37-9d77-9342f85226fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.151091 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645617bd-ce0b-4e37-9d77-9342f85226fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.151765 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645617bd-ce0b-4e37-9d77-9342f85226fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:44.153460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645617bd-ce0b-4e37-9d77-9342f85226fc admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:44.168014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1026/2060] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:44.178742 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645617bd-ce0b-4e37-9d77-9342f85226fc admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:44.180978 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-645617bd-ce0b-4e37-9d77-9342f85226fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:44.213663 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1035/2061] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:44.221495 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645617bd-ce0b-4e37-9d77-9342f85226fc admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:44.228195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed47e012-80b7-4e70-a920-8123325f82f0 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:44.229926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ed47e012-80b7-4e70-a920-8123325f82f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:44.233966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80513afb-35d2-4cfb-9c6b-20de05e80cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:44.235143 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80513afb-35d2-4cfb-9c6b-20de05e80cfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.235761 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80513afb-35d2-4cfb-9c6b-20de05e80cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.236827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80513afb-35d2-4cfb-9c6b-20de05e80cfa admin admin] PATH_INFO: `/v3/projects/e0fdde3b41fe4275b075be5bdc72aeac/users/cf0be44f38e6483785965a713b56f7dc/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.272359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80513afb-35d2-4cfb-9c6b-20de05e80cfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0fdde3b41fe4275b075be5bdc72aeac, user_id=cf0be44f38e6483785965a713b56f7dc, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:44.280183 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80513afb-35d2-4cfb-9c6b-20de05e80cfa admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:44.327273 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1027/2062] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:44 2022] PUT /identity/v3/projects/e0fdde3b41fe4275b075be5bdc72aeac/users/cf0be44f38e6483785965a713b56f7dc/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:44.329315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80513afb-35d2-4cfb-9c6b-20de05e80cfa admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:44.342045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed47e012-80b7-4e70-a920-8123325f82f0 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:44.423931 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb1ba2-a34b-46ef-a1ad-9d2b51213a6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:44.425160 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb1ba2-a34b-46ef-a1ad-9d2b51213a6c service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.425664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb1ba2-a34b-46ef-a1ad-9d2b51213a6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.426260 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb1ba2-a34b-46ef-a1ad-9d2b51213a6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.446330 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:44.451974 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:44.474519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80513afb-35d2-4cfb-9c6b-20de05e80cfa admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:44.482029 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5c810-8f9a-4641-be6d-ac0db68dacf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:44.482882 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5c810-8f9a-4641-be6d-ac0db68dacf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.483206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5c810-8f9a-4641-be6d-ac0db68dacf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.483488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5c810-8f9a-4641-be6d-ac0db68dacf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.484493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5c810-8f9a-4641-be6d-ac0db68dacf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:44.487727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5c810-8f9a-4641-be6d-ac0db68dacf0 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:44.509861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1028/2063] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:44.514898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5c810-8f9a-4641-be6d-ac0db68dacf0 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:44.562693 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5c810-8f9a-4641-be6d-ac0db68dacf0 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:44.583409 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb1ba2-a34b-46ef-a1ad-9d2b51213a6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:44.585553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cac906-83be-4278-9274-1dc2d6fbbe3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:44.586389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb1ba2-a34b-46ef-a1ad-9d2b51213a6c service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:44.601862 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cac906-83be-4278-9274-1dc2d6fbbe3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.602271 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cac906-83be-4278-9274-1dc2d6fbbe3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.602562 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cac906-83be-4278-9274-1dc2d6fbbe3e admin admin] PATH_INFO: `/v3/projects/e0fdde3b41fe4275b075be5bdc72aeac/users/cf0be44f38e6483785965a713b56f7dc/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.609622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cac906-83be-4278-9274-1dc2d6fbbe3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0fdde3b41fe4275b075be5bdc72aeac, user_id=cf0be44f38e6483785965a713b56f7dc, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:44.622017 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1036/2064] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:44.634368 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cac906-83be-4278-9274-1dc2d6fbbe3e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:44.697120 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1029/2065] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:44 2022] PUT /identity/v3/projects/e0fdde3b41fe4275b075be5bdc72aeac/users/cf0be44f38e6483785965a713b56f7dc/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:44.712097 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9674d-6412-4ae1-977b-91c99b4967af None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:44.712490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9674d-6412-4ae1-977b-91c99b4967af None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:44.713001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9674d-6412-4ae1-977b-91c99b4967af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:44.786493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-dee9674d-6412-4ae1-977b-91c99b4967af None None] MFA Rules not processed for user `cf0be44f38e6483785965a713b56f7dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:44.791127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-dee9674d-6412-4ae1-977b-91c99b4967af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:45.099939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1037/2066] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:45.158887 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cac906-83be-4278-9274-1dc2d6fbbe3e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:45.161879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-43cac906-83be-4278-9274-1dc2d6fbbe3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:45.497759 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cac906-83be-4278-9274-1dc2d6fbbe3e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:45.506737 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed51b8-afb6-4d6d-80a2-fe58655444bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:45.507824 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed51b8-afb6-4d6d-80a2-fe58655444bc service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:45.508494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed51b8-afb6-4d6d-80a2-fe58655444bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:45.509000 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed51b8-afb6-4d6d-80a2-fe58655444bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:45.531181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed51b8-afb6-4d6d-80a2-fe58655444bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:45.534949 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed51b8-afb6-4d6d-80a2-fe58655444bc service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:45.556677 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1030/2067] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:45 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:15:45.772101 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee9674d-6412-4ae1-977b-91c99b4967af None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:45.802037 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee9674d-6412-4ae1-977b-91c99b4967af None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:45.817454 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f567fca-bf3c-46cb-819a-f1e975ce243e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:45.818527 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f567fca-bf3c-46cb-819a-f1e975ce243e service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:45.819046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f567fca-bf3c-46cb-819a-f1e975ce243e service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:45.820053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f567fca-bf3c-46cb-819a-f1e975ce243e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:45.826588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed51b8-afb6-4d6d-80a2-fe58655444bc service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:45.829190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d9ed51b8-afb6-4d6d-80a2-fe58655444bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:45.842217 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6f567fca-bf3c-46cb-819a-f1e975ce243e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:45.992560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed51b8-afb6-4d6d-80a2-fe58655444bc service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:46.000319 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba0e51-1bcb-48f1-9e01-ec00bfd8bce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:46.000879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba0e51-1bcb-48f1-9e01-ec00bfd8bce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.001686 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba0e51-1bcb-48f1-9e01-ec00bfd8bce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.002023 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba0e51-1bcb-48f1-9e01-ec00bfd8bce3 admin admin] PATH_INFO: `/v3/projects/7778fdf4a6304f129177bff854f2922c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.003513 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba0e51-1bcb-48f1-9e01-ec00bfd8bce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7778fdf4a6304f129177bff854f2922c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:46.006109 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba0e51-1bcb-48f1-9e01-ec00bfd8bce3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:46.032102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f567fca-bf3c-46cb-819a-f1e975ce243e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:46.035095 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f567fca-bf3c-46cb-819a-f1e975ce243e service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:46.047268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1038/2068] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:15:46.048883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f567fca-bf3c-46cb-819a-f1e975ce243e service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:46.050488 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6f567fca-bf3c-46cb-819a-f1e975ce243e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:46.066200 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f567fca-bf3c-46cb-819a-f1e975ce243e service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:46.071967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b7beb-885f-4741-8c8d-c5fe6436ccf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:46.072735 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b7beb-885f-4741-8c8d-c5fe6436ccf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.073015 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b7beb-885f-4741-8c8d-c5fe6436ccf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.073343 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b7beb-885f-4741-8c8d-c5fe6436ccf1 admin admin] PATH_INFO: `/v3/projects/4f48e578ec97412a9eb3666595c4743f` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.074779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b7beb-885f-4741-8c8d-c5fe6436ccf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f48e578ec97412a9eb3666595c4743f)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:46.076741 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b7beb-885f-4741-8c8d-c5fe6436ccf1 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:46.108439 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:46.109635 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f1ba0e51-1bcb-48f1-9e01-ec00bfd8bce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7778fdf4a6304f129177bff854f2922c', 'request_id': 'req-f1ba0e51-1bcb-48f1-9e01-ec00bfd8bce3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:46.123332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f1ba0e51-1bcb-48f1-9e01-ec00bfd8bce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7778fdf4a6304f129177bff854f2922c', 'request_id': 'req-f1ba0e51-1bcb-48f1-9e01-ec00bfd8bce3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:46.133510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f1ba0e51-1bcb-48f1-9e01-ec00bfd8bce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7778fdf4a6304f129177bff854f2922c', 'request_id': 'req-f1ba0e51-1bcb-48f1-9e01-ec00bfd8bce3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:46.142828 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-200b7beb-885f-4741-8c8d-c5fe6436ccf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f48e578ec97412a9eb3666595c4743f', 'request_id': 'req-200b7beb-885f-4741-8c8d-c5fe6436ccf1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:46.151791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-200b7beb-885f-4741-8c8d-c5fe6436ccf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f48e578ec97412a9eb3666595c4743f', 'request_id': 'req-200b7beb-885f-4741-8c8d-c5fe6436ccf1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:46.154901 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f1ba0e51-1bcb-48f1-9e01-ec00bfd8bce3 admin admin] The token cache is being invalidate because project 7778fdf4a6304f129177bff854f2922c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:46.155178 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f1ba0e51-1bcb-48f1-9e01-ec00bfd8bce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:46.155843 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f1ba0e51-1bcb-48f1-9e01-ec00bfd8bce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:46.157570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1031/2069] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:45 2022] DELETE /identity/v3/projects/7778fdf4a6304f129177bff854f2922c => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:46.163607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821584f-ed85-495e-904e-0b6361a3ac37 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.164047 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821584f-ed85-495e-904e-0b6361a3ac37 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.164493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821584f-ed85-495e-904e-0b6361a3ac37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.168091 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-200b7beb-885f-4741-8c8d-c5fe6436ccf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f48e578ec97412a9eb3666595c4743f', 'request_id': 'req-200b7beb-885f-4741-8c8d-c5fe6436ccf1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:46.180146 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:46.187537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-200b7beb-885f-4741-8c8d-c5fe6436ccf1 admin admin] The token cache is being invalidate because project 4f48e578ec97412a9eb3666595c4743f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:46.191008 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-200b7beb-885f-4741-8c8d-c5fe6436ccf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:46.192465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-200b7beb-885f-4741-8c8d-c5fe6436ccf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:46.196155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1039/2070] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:46 2022] DELETE /identity/v3/projects/4f48e578ec97412a9eb3666595c4743f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:46.197780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d821584f-ed85-495e-904e-0b6361a3ac37 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:46.201258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d821584f-ed85-495e-904e-0b6361a3ac37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:46.213104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afff0af-e2bb-4b63-a1a7-9f44a78e070e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.213579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afff0af-e2bb-4b63-a1a7-9f44a78e070e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.214039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afff0af-e2bb-4b63-a1a7-9f44a78e070e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.260303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8afff0af-e2bb-4b63-a1a7-9f44a78e070e None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:46.261456 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8afff0af-e2bb-4b63-a1a7-9f44a78e070e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:46.313479 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1032/2071] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:46.328930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a991635-7c29-4f22-8fbf-d8d44fae762c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.330054 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a991635-7c29-4f22-8fbf-d8d44fae762c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.330560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a991635-7c29-4f22-8fbf-d8d44fae762c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.357074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1a991635-7c29-4f22-8fbf-d8d44fae762c None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:46.358866 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1a991635-7c29-4f22-8fbf-d8d44fae762c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:46.372347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1033/2072] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:15:46.376245 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1040/2073] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:46.382887 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a991635-7c29-4f22-8fbf-d8d44fae762c None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:46.389790 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a991635-7c29-4f22-8fbf-d8d44fae762c None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:46.390090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30b98d-7bc1-4f20-8195-848a6f8d8637 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.390363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30b98d-7bc1-4f20-8195-848a6f8d8637 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.390908 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30b98d-7bc1-4f20-8195-848a6f8d8637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.396206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346b45b-ecc8-4b3e-853a-a9413488d985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:46.397035 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346b45b-ecc8-4b3e-853a-a9413488d985 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.397534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346b45b-ecc8-4b3e-853a-a9413488d985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.397708 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346b45b-ecc8-4b3e-853a-a9413488d985 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.398558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346b45b-ecc8-4b3e-853a-a9413488d985 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:46.398995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346b45b-ecc8-4b3e-853a-a9413488d985 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:46.400871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346b45b-ecc8-4b3e-853a-a9413488d985 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:46.425674 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2a30b98d-7bc1-4f20-8195-848a6f8d8637 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:46.425965 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1034/2074] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:46.427960 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2a30b98d-7bc1-4f20-8195-848a6f8d8637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:46.432813 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346b45b-ecc8-4b3e-853a-a9413488d985 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:46.449646 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1041/2075] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:15:46.451081 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30b98d-7bc1-4f20-8195-848a6f8d8637 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:46.464265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346b45b-ecc8-4b3e-853a-a9413488d985 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:46.472120 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c57774-5dac-4982-9583-99cbdb6a491f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:46.472748 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c57774-5dac-4982-9583-99cbdb6a491f admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.473097 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c57774-5dac-4982-9583-99cbdb6a491f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.473591 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c57774-5dac-4982-9583-99cbdb6a491f admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.474482 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c57774-5dac-4982-9583-99cbdb6a491f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:46.474827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30b98d-7bc1-4f20-8195-848a6f8d8637 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:46.476308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c57774-5dac-4982-9583-99cbdb6a491f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:46.490433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e129f2c-4b10-47a1-8850-9acb962289a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:46.491816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e129f2c-4b10-47a1-8850-9acb962289a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.492653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e129f2c-4b10-47a1-8850-9acb962289a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.493299 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e129f2c-4b10-47a1-8850-9acb962289a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.494796 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e129f2c-4b10-47a1-8850-9acb962289a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:46.495466 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e129f2c-4b10-47a1-8850-9acb962289a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:46.500683 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e129f2c-4b10-47a1-8850-9acb962289a1 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:46.514510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1035/2076] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:46 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:46.530759 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1042/2077] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:46.535454 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c57774-5dac-4982-9583-99cbdb6a491f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:46.535629 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e129f2c-4b10-47a1-8850-9acb962289a1 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:46.553555 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e129f2c-4b10-47a1-8850-9acb962289a1 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:46.645076 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c57774-5dac-4982-9583-99cbdb6a491f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:46.652490 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cbd6a-4104-4530-8967-51c8de828138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:46.653230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cbd6a-4104-4530-8967-51c8de828138 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.653564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cbd6a-4104-4530-8967-51c8de828138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.653898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cbd6a-4104-4530-8967-51c8de828138 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.654695 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cbd6a-4104-4530-8967-51c8de828138 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:46.657110 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cbd6a-4104-4530-8967-51c8de828138 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:46.661809 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4edbbf-cefb-46fe-9874-b5305a86c73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:46.662974 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4edbbf-cefb-46fe-9874-b5305a86c73b admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.663466 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4edbbf-cefb-46fe-9874-b5305a86c73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.663939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4edbbf-cefb-46fe-9874-b5305a86c73b admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.665090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4edbbf-cefb-46fe-9874-b5305a86c73b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:46.669072 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4edbbf-cefb-46fe-9874-b5305a86c73b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:46.690298 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1036/2078] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:46 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:46.697136 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cbd6a-4104-4530-8967-51c8de828138 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:46.717107 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cbd6a-4104-4530-8967-51c8de828138 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:46.730511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1043/2079] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:46 2022] POST /identity/v3/projects => generated 377 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:15:46.731897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df328def-c714-4102-80cf-315f8a612568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:46.732777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4edbbf-cefb-46fe-9874-b5305a86c73b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:46.732905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df328def-c714-4102-80cf-315f8a612568 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.733262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df328def-c714-4102-80cf-315f8a612568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.733753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df328def-c714-4102-80cf-315f8a612568 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.734612 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df328def-c714-4102-80cf-315f8a612568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:46.740010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df328def-c714-4102-80cf-315f8a612568 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:46.756611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1037/2080] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:15:46.763921 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df328def-c714-4102-80cf-315f8a612568 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:46.776515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:46.784045 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df328def-c714-4102-80cf-315f8a612568 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:46.864026 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:46.874757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4edbbf-cefb-46fe-9874-b5305a86c73b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:46.883867 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867d8e5-c245-46ca-9205-8405d32c1791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:46.884946 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867d8e5-c245-46ca-9205-8405d32c1791 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.885451 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867d8e5-c245-46ca-9205-8405d32c1791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.885938 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867d8e5-c245-46ca-9205-8405d32c1791 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.887789 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9867d8e5-c245-46ca-9205-8405d32c1791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:46.891111 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9867d8e5-c245-46ca-9205-8405d32c1791 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:46.893677 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e9f7e-24f2-401c-9025-7eda49dd2ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:46.894301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e9f7e-24f2-401c-9025-7eda49dd2ecf admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.894578 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e9f7e-24f2-401c-9025-7eda49dd2ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.894899 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e9f7e-24f2-401c-9025-7eda49dd2ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.895690 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e9f7e-24f2-401c-9025-7eda49dd2ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:46.897226 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e9f7e-24f2-401c-9025-7eda49dd2ecf admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:46.909588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1038/2081] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:46.916339 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e9f7e-24f2-401c-9025-7eda49dd2ecf admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:46.930763 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1044/2082] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:46 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:15:46.936550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e9f7e-24f2-401c-9025-7eda49dd2ecf admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:46.936746 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867d8e5-c245-46ca-9205-8405d32c1791 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:46.945562 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a225326f-8bec-4022-9860-740fbdafd94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:46.946253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225326f-8bec-4022-9860-740fbdafd94c admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.946537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225326f-8bec-4022-9860-740fbdafd94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.946871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225326f-8bec-4022-9860-740fbdafd94c admin admin] PATH_INFO: `/v3/projects/ea6f812fb0e74be3a827cc8d47d54ad2/users/8354f49315884f9a888dd6231ff1a44a/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.952610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867d8e5-c245-46ca-9205-8405d32c1791 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:46.960702 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fbece-50e0-4426-8d10-1682fb304da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:46.961636 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fbece-50e0-4426-8d10-1682fb304da5 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:46.961948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fbece-50e0-4426-8d10-1682fb304da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:46.962341 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fbece-50e0-4426-8d10-1682fb304da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:46.963343 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01fbece-50e0-4426-8d10-1682fb304da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:46.966247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01fbece-50e0-4426-8d10-1682fb304da5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:46.971166 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225326f-8bec-4022-9860-740fbdafd94c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea6f812fb0e74be3a827cc8d47d54ad2, user_id=8354f49315884f9a888dd6231ff1a44a, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:46.974885 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225326f-8bec-4022-9860-740fbdafd94c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:46.983400 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1045/2083] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:46.988922 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fbece-50e0-4426-8d10-1682fb304da5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:46.997566 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1039/2084] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:46 2022] PUT /identity/v3/projects/ea6f812fb0e74be3a827cc8d47d54ad2/users/8354f49315884f9a888dd6231ff1a44a/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:47.000945 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fbece-50e0-4426-8d10-1682fb304da5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:47.010296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2c93d-70f2-4455-922b-87c23e6b12e8 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:47.011284 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2c93d-70f2-4455-922b-87c23e6b12e8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:47.011741 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2c93d-70f2-4455-922b-87c23e6b12e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:47.061447 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-c7a2c93d-70f2-4455-922b-87c23e6b12e8 None None] MFA Rules not processed for user `8354f49315884f9a888dd6231ff1a44a`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:47.064575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c7a2c93d-70f2-4455-922b-87c23e6b12e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:47.120299 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929c0b5-62ba-4e24-8463-da7f8ee50beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:47.121018 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929c0b5-62ba-4e24-8463-da7f8ee50beb admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:47.121497 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929c0b5-62ba-4e24-8463-da7f8ee50beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:47.121855 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929c0b5-62ba-4e24-8463-da7f8ee50beb admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:47.122968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929c0b5-62ba-4e24-8463-da7f8ee50beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:47.125065 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929c0b5-62ba-4e24-8463-da7f8ee50beb admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:47.143183 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1046/2085] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:15:47.143183 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929c0b5-62ba-4e24-8463-da7f8ee50beb admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:47.157426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929c0b5-62ba-4e24-8463-da7f8ee50beb admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:47.163377 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d24b8-79ea-44d7-b092-ba426b6c071c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:47.163851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d24b8-79ea-44d7-b092-ba426b6c071c admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:47.164391 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d24b8-79ea-44d7-b092-ba426b6c071c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:47.164728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d24b8-79ea-44d7-b092-ba426b6c071c admin admin] PATH_INFO: `/v3/projects/ac737b893dca4bbeaadfa9112415bb4f/users/ade45b21943d4dd283dcdd1b450561d4/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:47.195072 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d24b8-79ea-44d7-b092-ba426b6c071c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac737b893dca4bbeaadfa9112415bb4f, user_id=ade45b21943d4dd283dcdd1b450561d4, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:47.197890 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d24b8-79ea-44d7-b092-ba426b6c071c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:47.198308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1040/2086] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:47.214653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2c93d-70f2-4455-922b-87c23e6b12e8 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:47.216654 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c7a2c93d-70f2-4455-922b-87c23e6b12e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:47.228359 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1047/2087] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:47 2022] PUT /identity/v3/projects/ac737b893dca4bbeaadfa9112415bb4f/users/ade45b21943d4dd283dcdd1b450561d4/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:47.232135 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec34840-7ffe-45c6-8987-d87d733419ac None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:47.232668 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec34840-7ffe-45c6-8987-d87d733419ac None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:47.233092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec34840-7ffe-45c6-8987-d87d733419ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:47.270979 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-3ec34840-7ffe-45c6-8987-d87d733419ac None None] MFA Rules not processed for user `ade45b21943d4dd283dcdd1b450561d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:47.272268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3ec34840-7ffe-45c6-8987-d87d733419ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:47.339090 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2c93d-70f2-4455-922b-87c23e6b12e8 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:47.357074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:47.364363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1048/2088] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:47.379155 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec34840-7ffe-45c6-8987-d87d733419ac None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:47.383905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3ec34840-7ffe-45c6-8987-d87d733419ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:47.409056 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:47.434165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cad353-bdac-4cfe-bc94-f2361640126b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50ef7db27744c7ebd2ec95cecc9d1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:47.435243 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cad353-bdac-4cfe-bc94-f2361640126b service glance] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:47.435727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cad353-bdac-4cfe-bc94-f2361640126b service glance] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:47.436207 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cad353-bdac-4cfe-bc94-f2361640126b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:47.456552 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec34840-7ffe-45c6-8987-d87d733419ac None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:47.464428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb04d3-7f01-43a1-9679-c46a95446a7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:47.464991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb04d3-7f01-43a1-9679-c46a95446a7d service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:47.465434 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb04d3-7f01-43a1-9679-c46a95446a7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:47.465903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb04d3-7f01-43a1-9679-c46a95446a7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:47.482714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb04d3-7f01-43a1-9679-c46a95446a7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:47.486249 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb04d3-7f01-43a1-9679-c46a95446a7d service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:47.503787 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1049/2089] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:47.530892 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cad353-bdac-4cfe-bc94-f2361640126b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:47.534077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cad353-bdac-4cfe-bc94-f2361640126b service glance] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:47.544323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1041/2090] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:52.778264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb04d3-7f01-43a1-9679-c46a95446a7d service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:52.779783 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-19fb04d3-7f01-43a1-9679-c46a95446a7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:52.874336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb04d3-7f01-43a1-9679-c46a95446a7d service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:52.881460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedde09-6963-4796-88b3-3486f7ba9c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:52.882568 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedde09-6963-4796-88b3-3486f7ba9c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:52.882985 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedde09-6963-4796-88b3-3486f7ba9c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:52.883618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedde09-6963-4796-88b3-3486f7ba9c33 admin admin] PATH_INFO: `/v3/users/76afe7068d604416a2621aee343f50b3` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:52.884982 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dedde09-6963-4796-88b3-3486f7ba9c33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76afe7068d604416a2621aee343f50b3)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:52.887131 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dedde09-6963-4796-88b3-3486f7ba9c33 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:52.960913 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6dedde09-6963-4796-88b3-3486f7ba9c33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76afe7068d604416a2621aee343f50b3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:52.974152 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6dedde09-6963-4796-88b3-3486f7ba9c33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76afe7068d604416a2621aee343f50b3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:52.990562 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6dedde09-6963-4796-88b3-3486f7ba9c33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76afe7068d604416a2621aee343f50b3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:52.991951 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6dedde09-6963-4796-88b3-3486f7ba9c33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76afe7068d604416a2621aee343f50b3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:53.005532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6dedde09-6963-4796-88b3-3486f7ba9c33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76afe7068d604416a2621aee343f50b3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:53.012083 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1050/2091] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:52 2022] DELETE /identity/v3/users/76afe7068d604416a2621aee343f50b3 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:53.017281 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00d9df-61ca-42ed-acef-3f9d137c1d3b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:53.017813 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00d9df-61ca-42ed-acef-3f9d137c1d3b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:53.018027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00d9df-61ca-42ed-acef-3f9d137c1d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:53.042759 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0d00d9df-61ca-42ed-acef-3f9d137c1d3b None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:53.046743 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0d00d9df-61ca-42ed-acef-3f9d137c1d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:53.112778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1042/2092] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:53.126909 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedde09-6963-4796-88b3-3486f7ba9c33 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:53.128767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6dedde09-6963-4796-88b3-3486f7ba9c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:53.156404 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:53.201783 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedde09-6963-4796-88b3-3486f7ba9c33 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:53.206515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6de03-a582-4e42-a469-ece12198d4bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:53.207397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6de03-a582-4e42-a469-ece12198d4bd service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:53.207683 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6de03-a582-4e42-a469-ece12198d4bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:53.207957 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6de03-a582-4e42-a469-ece12198d4bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:53.217878 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6de03-a582-4e42-a469-ece12198d4bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:53.219440 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6de03-a582-4e42-a469-ece12198d4bd service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:53.230018 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1051/2093] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:53.809409 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00d9df-61ca-42ed-acef-3f9d137c1d3b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:53.810978 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0d00d9df-61ca-42ed-acef-3f9d137c1d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:53.825922 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00d9df-61ca-42ed-acef-3f9d137c1d3b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:53.844567 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bce813-5981-4d33-8d47-1c7ecc2115fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:53.846558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bce813-5981-4d33-8d47-1c7ecc2115fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:53.847020 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bce813-5981-4d33-8d47-1c7ecc2115fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:53.847510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bce813-5981-4d33-8d47-1c7ecc2115fb admin admin] PATH_INFO: `/v3/projects/15cf0fb91c77486488120f518735f991` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:53.849803 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bce813-5981-4d33-8d47-1c7ecc2115fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=15cf0fb91c77486488120f518735f991)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:53.854446 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bce813-5981-4d33-8d47-1c7ecc2115fb admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:53.968996 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-27bce813-5981-4d33-8d47-1c7ecc2115fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15cf0fb91c77486488120f518735f991', 'request_id': 'req-27bce813-5981-4d33-8d47-1c7ecc2115fb'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:53.987090 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-27bce813-5981-4d33-8d47-1c7ecc2115fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15cf0fb91c77486488120f518735f991', 'request_id': 'req-27bce813-5981-4d33-8d47-1c7ecc2115fb'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:53.997691 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-27bce813-5981-4d33-8d47-1c7ecc2115fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15cf0fb91c77486488120f518735f991', 'request_id': 'req-27bce813-5981-4d33-8d47-1c7ecc2115fb'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:54.017910 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-27bce813-5981-4d33-8d47-1c7ecc2115fb admin admin] The token cache is being invalidate because project 15cf0fb91c77486488120f518735f991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:54.019094 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-27bce813-5981-4d33-8d47-1c7ecc2115fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:54.019094 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-27bce813-5981-4d33-8d47-1c7ecc2115fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:54.025209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1043/2094] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:53 2022] DELETE /identity/v3/projects/15cf0fb91c77486488120f518735f991 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:54.029391 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6de03-a582-4e42-a469-ece12198d4bd service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:54.041779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6de03-a582-4e42-a469-ece12198d4bd service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:54.124732 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8420f-8076-4414-bd0f-f846291be718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:54.125616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8420f-8076-4414-bd0f-f846291be718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:54.125962 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8420f-8076-4414-bd0f-f846291be718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:54.127655 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8420f-8076-4414-bd0f-f846291be718 admin admin] PATH_INFO: `/v3/users/cf0be44f38e6483785965a713b56f7dc` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:54.129576 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8420f-8076-4414-bd0f-f846291be718 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf0be44f38e6483785965a713b56f7dc)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:54.132263 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8420f-8076-4414-bd0f-f846291be718 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:54.208903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cac8420f-8076-4414-bd0f-f846291be718 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf0be44f38e6483785965a713b56f7dc'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:54.216782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cac8420f-8076-4414-bd0f-f846291be718 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf0be44f38e6483785965a713b56f7dc'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:54.230096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cac8420f-8076-4414-bd0f-f846291be718 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf0be44f38e6483785965a713b56f7dc'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:54.231023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cac8420f-8076-4414-bd0f-f846291be718 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf0be44f38e6483785965a713b56f7dc'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:54.239927 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cac8420f-8076-4414-bd0f-f846291be718 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf0be44f38e6483785965a713b56f7dc'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:54.246795 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1052/2095] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:54 2022] DELETE /identity/v3/users/cf0be44f38e6483785965a713b56f7dc => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:54.584596 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bce813-5981-4d33-8d47-1c7ecc2115fb admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:54.587152 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-27bce813-5981-4d33-8d47-1c7ecc2115fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:54.611517 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bce813-5981-4d33-8d47-1c7ecc2115fb admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:54.628514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:54.716696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66dcc1-5951-4893-9721-f827c597922f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:54.717345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66dcc1-5951-4893-9721-f827c597922f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:54.717614 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66dcc1-5951-4893-9721-f827c597922f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:54.717882 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66dcc1-5951-4893-9721-f827c597922f admin admin] PATH_INFO: `/v3/projects/e0fdde3b41fe4275b075be5bdc72aeac` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:54.719076 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66dcc1-5951-4893-9721-f827c597922f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0fdde3b41fe4275b075be5bdc72aeac)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:54.721584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66dcc1-5951-4893-9721-f827c597922f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:54.884829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ef66dcc1-5951-4893-9721-f827c597922f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0fdde3b41fe4275b075be5bdc72aeac', 'request_id': 'req-ef66dcc1-5951-4893-9721-f827c597922f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:54.900174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ef66dcc1-5951-4893-9721-f827c597922f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0fdde3b41fe4275b075be5bdc72aeac', 'request_id': 'req-ef66dcc1-5951-4893-9721-f827c597922f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:54.913081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ef66dcc1-5951-4893-9721-f827c597922f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0fdde3b41fe4275b075be5bdc72aeac', 'request_id': 'req-ef66dcc1-5951-4893-9721-f827c597922f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:54.927745 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ef66dcc1-5951-4893-9721-f827c597922f admin admin] The token cache is being invalidate because project e0fdde3b41fe4275b075be5bdc72aeac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:54.928429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ef66dcc1-5951-4893-9721-f827c597922f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:54.929259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ef66dcc1-5951-4893-9721-f827c597922f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:54.931412 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1044/2096] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:54 2022] DELETE /identity/v3/projects/e0fdde3b41fe4275b075be5bdc72aeac => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:54.938283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e84229-b66e-424a-9add-b8c2b28c9bea None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:54.939219 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e84229-b66e-424a-9add-b8c2b28c9bea None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:54.939964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e84229-b66e-424a-9add-b8c2b28c9bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:54.967985 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f8e84229-b66e-424a-9add-b8c2b28c9bea None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:54.969369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f8e84229-b66e-424a-9add-b8c2b28c9bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:55.044471 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:55.084404 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1053/2097] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:15:55.094605 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b1a18-c94a-40da-8d3b-a5cfa6ac4eb6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:55.096166 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b1a18-c94a-40da-8d3b-a5cfa6ac4eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:55.096166 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b1a18-c94a-40da-8d3b-a5cfa6ac4eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:55.121376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-384b1a18-c94a-40da-8d3b-a5cfa6ac4eb6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:55.122714 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-384b1a18-c94a-40da-8d3b-a5cfa6ac4eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:55.142726 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1045/2098] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:15:55.145553 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e84229-b66e-424a-9add-b8c2b28c9bea None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:55.161278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e84229-b66e-424a-9add-b8c2b28c9bea None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:55.167269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c8ec6-ce9f-436a-a5e1-f299411d801f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:55.168154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c8ec6-ce9f-436a-a5e1-f299411d801f admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:55.168529 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c8ec6-ce9f-436a-a5e1-f299411d801f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:55.168897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c8ec6-ce9f-436a-a5e1-f299411d801f admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:55.169948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c8ec6-ce9f-436a-a5e1-f299411d801f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:55.170432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c8ec6-ce9f-436a-a5e1-f299411d801f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:55.172337 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c8ec6-ce9f-436a-a5e1-f299411d801f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:55.190013 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1054/2099] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:15:55.193593 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b1a18-c94a-40da-8d3b-a5cfa6ac4eb6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:55.221447 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b1a18-c94a-40da-8d3b-a5cfa6ac4eb6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:55.230290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49ab22-6e00-4ad6-b9cd-c2379f13b4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:55.232469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49ab22-6e00-4ad6-b9cd-c2379f13b4e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:55.232469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49ab22-6e00-4ad6-b9cd-c2379f13b4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:55.232891 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49ab22-6e00-4ad6-b9cd-c2379f13b4e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:55.234115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b49ab22-6e00-4ad6-b9cd-c2379f13b4e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:55.238624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b49ab22-6e00-4ad6-b9cd-c2379f13b4e2 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:55.294798 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1046/2100] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:55 2022] POST /identity/v3/projects => generated 389 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:55.296502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c8ec6-ce9f-436a-a5e1-f299411d801f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:55.312638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c8ec6-ce9f-436a-a5e1-f299411d801f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:55.436040 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d4cfb-b27d-4c96-badf-d55b463e09c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:55.436675 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d4cfb-b27d-4c96-badf-d55b463e09c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:55.436955 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d4cfb-b27d-4c96-badf-d55b463e09c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:55.437222 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d4cfb-b27d-4c96-badf-d55b463e09c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:55.437870 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d4cfb-b27d-4c96-badf-d55b463e09c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:55.442268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d4cfb-b27d-4c96-badf-d55b463e09c1 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:55.474736 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1055/2101] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:55 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:55.477564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49ab22-6e00-4ad6-b9cd-c2379f13b4e2 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:55.489465 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49ab22-6e00-4ad6-b9cd-c2379f13b4e2 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:55.497039 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144ecfb-9992-4499-af02-b9f826503937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:55.497948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144ecfb-9992-4499-af02-b9f826503937 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:55.498223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144ecfb-9992-4499-af02-b9f826503937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:55.498489 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144ecfb-9992-4499-af02-b9f826503937 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:55.499166 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144ecfb-9992-4499-af02-b9f826503937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:55.500766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144ecfb-9992-4499-af02-b9f826503937 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:55.515931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1047/2102] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:55.520860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d4cfb-b27d-4c96-badf-d55b463e09c1 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:55.531865 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d4cfb-b27d-4c96-badf-d55b463e09c1 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:55.539636 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763b56d-0658-48fb-beee-927893335934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:55.540322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763b56d-0658-48fb-beee-927893335934 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:55.540604 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763b56d-0658-48fb-beee-927893335934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:55.540879 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763b56d-0658-48fb-beee-927893335934 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:55.541706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e763b56d-0658-48fb-beee-927893335934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:55.543552 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e763b56d-0658-48fb-beee-927893335934 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:55.560174 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1056/2103] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:55.566372 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144ecfb-9992-4499-af02-b9f826503937 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:55.577833 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144ecfb-9992-4499-af02-b9f826503937 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:55.583613 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b95427-421e-4c7f-bf6c-48c7ac4dfa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:55.584291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b95427-421e-4c7f-bf6c-48c7ac4dfa90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:55.584583 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b95427-421e-4c7f-bf6c-48c7ac4dfa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:55.585002 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b95427-421e-4c7f-bf6c-48c7ac4dfa90 admin admin] PATH_INFO: `/v3/projects/8342bc00440947079a697e3babfb1a5f/users/891d422197274d81b8386ced44b19afd/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:55.604040 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b95427-421e-4c7f-bf6c-48c7ac4dfa90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8342bc00440947079a697e3babfb1a5f, user_id=891d422197274d81b8386ced44b19afd, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:55.606954 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b95427-421e-4c7f-bf6c-48c7ac4dfa90 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:55.627501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1048/2104] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:55 2022] PUT /identity/v3/projects/8342bc00440947079a697e3babfb1a5f/users/891d422197274d81b8386ced44b19afd/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:55.638220 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e3912-492c-401f-95c8-e40b391e562a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:55.638501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e3912-492c-401f-95c8-e40b391e562a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:55.638804 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e3912-492c-401f-95c8-e40b391e562a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:55.671311 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e09e3912-492c-401f-95c8-e40b391e562a None None] MFA Rules not processed for user `891d422197274d81b8386ced44b19afd`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:55.672574 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e09e3912-492c-401f-95c8-e40b391e562a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:55.752497 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1057/2105] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:55.757252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b95427-421e-4c7f-bf6c-48c7ac4dfa90 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:55.811793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:55.840739 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b95427-421e-4c7f-bf6c-48c7ac4dfa90 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:55.847578 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79014cfd-224d-41c4-a186-f1de3d3b02bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:55.848208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79014cfd-224d-41c4-a186-f1de3d3b02bf admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:55.848483 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79014cfd-224d-41c4-a186-f1de3d3b02bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:55.849643 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79014cfd-224d-41c4-a186-f1de3d3b02bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:55.849643 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79014cfd-224d-41c4-a186-f1de3d3b02bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:55.852467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79014cfd-224d-41c4-a186-f1de3d3b02bf admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:55.901856 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1049/2106] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:55 2022] POST /identity/v3/projects => generated 389 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:55.903254 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e3912-492c-401f-95c8-e40b391e562a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:55.918843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e3912-492c-401f-95c8-e40b391e562a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:55.974306 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:56.008674 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18382a0-4f00-4bdc-a9cc-96551f3075ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:56.009308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18382a0-4f00-4bdc-a9cc-96551f3075ec admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:56.009587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18382a0-4f00-4bdc-a9cc-96551f3075ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:56.009859 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18382a0-4f00-4bdc-a9cc-96551f3075ec admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:56.010826 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18382a0-4f00-4bdc-a9cc-96551f3075ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:56.012770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18382a0-4f00-4bdc-a9cc-96551f3075ec admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:56.046314 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79014cfd-224d-41c4-a186-f1de3d3b02bf admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:56.046836 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1058/2107] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:55 2022] POST /identity/v3/users => generated 301 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:15:56.058628 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79014cfd-224d-41c4-a186-f1de3d3b02bf admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:56.064481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01e794-8722-4eae-a63f-2dc497de027a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:56.065350 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01e794-8722-4eae-a63f-2dc497de027a admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:56.065540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01e794-8722-4eae-a63f-2dc497de027a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:56.065762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01e794-8722-4eae-a63f-2dc497de027a admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:56.066435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01e794-8722-4eae-a63f-2dc497de027a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:56.068103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01e794-8722-4eae-a63f-2dc497de027a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:56.092575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1050/2108] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:15:56.096368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18382a0-4f00-4bdc-a9cc-96551f3075ec admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:56.107567 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18382a0-4f00-4bdc-a9cc-96551f3075ec admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:56.112681 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667cfff-0ad9-489b-96b6-47736f4b2a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:56.113586 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667cfff-0ad9-489b-96b6-47736f4b2a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:56.113586 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667cfff-0ad9-489b-96b6-47736f4b2a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:56.114402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667cfff-0ad9-489b-96b6-47736f4b2a83 admin admin] PATH_INFO: `/v3/projects/00555db2cc544f0496e7a191f4805160/users/f69d6f2f215e4663bf4d53582005f017/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:56.143045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2667cfff-0ad9-489b-96b6-47736f4b2a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00555db2cc544f0496e7a191f4805160, user_id=f69d6f2f215e4663bf4d53582005f017, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:56.143045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2667cfff-0ad9-489b-96b6-47736f4b2a83 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:56.159344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1059/2109] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:56 2022] PUT /identity/v3/projects/00555db2cc544f0496e7a191f4805160/users/f69d6f2f215e4663bf4d53582005f017/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:56.165252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01e794-8722-4eae-a63f-2dc497de027a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:56.176471 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01e794-8722-4eae-a63f-2dc497de027a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:56.237962 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a685f-bc59-4db3-b6ef-08bdc5e0dd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:56.238582 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a685f-bc59-4db3-b6ef-08bdc5e0dd67 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:56.238897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a685f-bc59-4db3-b6ef-08bdc5e0dd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:56.239269 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a685f-bc59-4db3-b6ef-08bdc5e0dd67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:56.239825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a685f-bc59-4db3-b6ef-08bdc5e0dd67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:56.241456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a685f-bc59-4db3-b6ef-08bdc5e0dd67 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:56.252723 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1051/2110] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:56.257020 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667cfff-0ad9-489b-96b6-47736f4b2a83 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:56.267882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667cfff-0ad9-489b-96b6-47736f4b2a83 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:56.273477 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f45e3-7cf5-453d-bd04-0e51d808401b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:56.274525 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f45e3-7cf5-453d-bd04-0e51d808401b admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:56.274968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f45e3-7cf5-453d-bd04-0e51d808401b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:56.275286 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f45e3-7cf5-453d-bd04-0e51d808401b admin admin] PATH_INFO: `/v3/projects/00555db2cc544f0496e7a191f4805160/users/f69d6f2f215e4663bf4d53582005f017/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:56.277911 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f45e3-7cf5-453d-bd04-0e51d808401b admin admin] RBAC: Authorizing `identity:create_grant(project_id=00555db2cc544f0496e7a191f4805160, user_id=f69d6f2f215e4663bf4d53582005f017, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:56.280978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f45e3-7cf5-453d-bd04-0e51d808401b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:56.292795 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1060/2111] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:56 2022] PUT /identity/v3/projects/00555db2cc544f0496e7a191f4805160/users/f69d6f2f215e4663bf4d53582005f017/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:56.305357 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c6996-98d0-45fa-ae2b-80f57575c068 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:56.305645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c6996-98d0-45fa-ae2b-80f57575c068 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:56.305950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c6996-98d0-45fa-ae2b-80f57575c068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:56.338642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f60c6996-98d0-45fa-ae2b-80f57575c068 None None] MFA Rules not processed for user `f69d6f2f215e4663bf4d53582005f017`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:56.339897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f60c6996-98d0-45fa-ae2b-80f57575c068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:56.418539 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1052/2112] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:56 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:56.430377 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f45e3-7cf5-453d-bd04-0e51d808401b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:56.432276 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b02f45e3-7cf5-453d-bd04-0e51d808401b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:56.500321 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f45e3-7cf5-453d-bd04-0e51d808401b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:56.504236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c0866-2c29-45d4-ae8f-6d393b3760fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:56.504853 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c0866-2c29-45d4-ae8f-6d393b3760fe service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:56.505126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c0866-2c29-45d4-ae8f-6d393b3760fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:56.505404 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c0866-2c29-45d4-ae8f-6d393b3760fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:56.515985 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c0866-2c29-45d4-ae8f-6d393b3760fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:56.517480 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c0866-2c29-45d4-ae8f-6d393b3760fe service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:56.528722 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1061/2113] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:56 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:56.747953 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c6996-98d0-45fa-ae2b-80f57575c068 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:56.810758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:56.838460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c6996-98d0-45fa-ae2b-80f57575c068 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:56.843084 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ec6ca-a679-4b37-b231-5b14ddab270c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:56.843721 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ec6ca-a679-4b37-b231-5b14ddab270c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:56.844000 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ec6ca-a679-4b37-b231-5b14ddab270c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:56.844285 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ec6ca-a679-4b37-b231-5b14ddab270c admin admin] PATH_INFO: `/v3/users/891d422197274d81b8386ced44b19afd` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:56.846490 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ec6ca-a679-4b37-b231-5b14ddab270c admin admin] RBAC: Authorizing `identity:delete_user(user_id=891d422197274d81b8386ced44b19afd)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:56.848446 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ec6ca-a679-4b37-b231-5b14ddab270c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:56.914539 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-887ec6ca-a679-4b37-b231-5b14ddab270c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '891d422197274d81b8386ced44b19afd'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:56.923488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-887ec6ca-a679-4b37-b231-5b14ddab270c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '891d422197274d81b8386ced44b19afd'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:56.937567 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-887ec6ca-a679-4b37-b231-5b14ddab270c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '891d422197274d81b8386ced44b19afd'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:56.938821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-887ec6ca-a679-4b37-b231-5b14ddab270c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '891d422197274d81b8386ced44b19afd'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:56.939950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-887ec6ca-a679-4b37-b231-5b14ddab270c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '891d422197274d81b8386ced44b19afd'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:56.958988 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1053/2114] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:56 2022] DELETE /identity/v3/users/891d422197274d81b8386ced44b19afd => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:56.965162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b7dba-0c87-449e-9fe6-5303b4f7da2d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:56.965766 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b7dba-0c87-449e-9fe6-5303b4f7da2d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:56.966279 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b7dba-0c87-449e-9fe6-5303b4f7da2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:56.993174 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-5b0b7dba-0c87-449e-9fe6-5303b4f7da2d None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:56.994442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5b0b7dba-0c87-449e-9fe6-5303b4f7da2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:57.062766 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1062/2115] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:57.073649 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ec6ca-a679-4b37-b231-5b14ddab270c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:57.075365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-887ec6ca-a679-4b37-b231-5b14ddab270c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:57.141177 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ec6ca-a679-4b37-b231-5b14ddab270c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:57.145758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c62b4-d287-4074-ae2e-f264d1d9dd21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:57.146384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c62b4-d287-4074-ae2e-f264d1d9dd21 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:57.146682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c62b4-d287-4074-ae2e-f264d1d9dd21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:57.146996 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c62b4-d287-4074-ae2e-f264d1d9dd21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:57.158062 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c62b4-d287-4074-ae2e-f264d1d9dd21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:57.159771 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c62b4-d287-4074-ae2e-f264d1d9dd21 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:57.170710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1054/2116] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:15:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:15:57.589601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b7dba-0c87-449e-9fe6-5303b4f7da2d None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:57.591952 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5b0b7dba-0c87-449e-9fe6-5303b4f7da2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:57.615176 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b7dba-0c87-449e-9fe6-5303b4f7da2d None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:57.621254 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fba76b-1bc9-4b27-b2ea-20b9f656e15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:57.622324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fba76b-1bc9-4b27-b2ea-20b9f656e15c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:57.622851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fba76b-1bc9-4b27-b2ea-20b9f656e15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:57.623350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fba76b-1bc9-4b27-b2ea-20b9f656e15c admin admin] PATH_INFO: `/v3/projects/8342bc00440947079a697e3babfb1a5f` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:57.625255 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fba76b-1bc9-4b27-b2ea-20b9f656e15c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8342bc00440947079a697e3babfb1a5f)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:57.629002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fba76b-1bc9-4b27-b2ea-20b9f656e15c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:57.695188 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:57.723809 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-87fba76b-1bc9-4b27-b2ea-20b9f656e15c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8342bc00440947079a697e3babfb1a5f', 'request_id': 'req-87fba76b-1bc9-4b27-b2ea-20b9f656e15c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:57.734798 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-87fba76b-1bc9-4b27-b2ea-20b9f656e15c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8342bc00440947079a697e3babfb1a5f', 'request_id': 'req-87fba76b-1bc9-4b27-b2ea-20b9f656e15c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:57.743812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-87fba76b-1bc9-4b27-b2ea-20b9f656e15c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8342bc00440947079a697e3babfb1a5f', 'request_id': 'req-87fba76b-1bc9-4b27-b2ea-20b9f656e15c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:57.762429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-87fba76b-1bc9-4b27-b2ea-20b9f656e15c admin admin] The token cache is being invalidate because project 8342bc00440947079a697e3babfb1a5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:57.762893 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-87fba76b-1bc9-4b27-b2ea-20b9f656e15c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:57.763744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-87fba76b-1bc9-4b27-b2ea-20b9f656e15c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:57.765441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1063/2117] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:57 2022] DELETE /identity/v3/projects/8342bc00440947079a697e3babfb1a5f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:57.771439 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c62b4-d287-4074-ae2e-f264d1d9dd21 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:57.789426 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c62b4-d287-4074-ae2e-f264d1d9dd21 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:57.868250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95231a01-628e-450a-b0b5-e560a5244db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:57.868961 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95231a01-628e-450a-b0b5-e560a5244db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:57.869247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95231a01-628e-450a-b0b5-e560a5244db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:57.870226 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95231a01-628e-450a-b0b5-e560a5244db3 admin admin] PATH_INFO: `/v3/users/f69d6f2f215e4663bf4d53582005f017` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:57.871590 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95231a01-628e-450a-b0b5-e560a5244db3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69d6f2f215e4663bf4d53582005f017)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:57.873811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95231a01-628e-450a-b0b5-e560a5244db3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:57.876942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:57.957980 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-95231a01-628e-450a-b0b5-e560a5244db3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69d6f2f215e4663bf4d53582005f017'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:57.972899 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-95231a01-628e-450a-b0b5-e560a5244db3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69d6f2f215e4663bf4d53582005f017'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:57.999764 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-95231a01-628e-450a-b0b5-e560a5244db3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69d6f2f215e4663bf4d53582005f017'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:58.000900 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-95231a01-628e-450a-b0b5-e560a5244db3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69d6f2f215e4663bf4d53582005f017'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:58.002919 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-95231a01-628e-450a-b0b5-e560a5244db3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69d6f2f215e4663bf4d53582005f017'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:58.020562 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1055/2118] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:15:57 2022] DELETE /identity/v3/users/f69d6f2f215e4663bf4d53582005f017 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:58.405430 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fba76b-1bc9-4b27-b2ea-20b9f656e15c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:58.408180 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-87fba76b-1bc9-4b27-b2ea-20b9f656e15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:58.425727 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fba76b-1bc9-4b27-b2ea-20b9f656e15c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:58.519871 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec77bf6-ee6e-4b6f-ab3e-2ea4dbf11db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:58.520412 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec77bf6-ee6e-4b6f-ab3e-2ea4dbf11db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:58.520690 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec77bf6-ee6e-4b6f-ab3e-2ea4dbf11db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:58.520954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec77bf6-ee6e-4b6f-ab3e-2ea4dbf11db8 admin admin] PATH_INFO: `/v3/projects/00555db2cc544f0496e7a191f4805160` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:58.522152 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec77bf6-ee6e-4b6f-ab3e-2ea4dbf11db8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00555db2cc544f0496e7a191f4805160)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:58.524182 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec77bf6-ee6e-4b6f-ab3e-2ea4dbf11db8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:58.614523 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-eec77bf6-ee6e-4b6f-ab3e-2ea4dbf11db8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00555db2cc544f0496e7a191f4805160', 'request_id': 'req-eec77bf6-ee6e-4b6f-ab3e-2ea4dbf11db8'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:58.621242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-eec77bf6-ee6e-4b6f-ab3e-2ea4dbf11db8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00555db2cc544f0496e7a191f4805160', 'request_id': 'req-eec77bf6-ee6e-4b6f-ab3e-2ea4dbf11db8'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:58.628026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-eec77bf6-ee6e-4b6f-ab3e-2ea4dbf11db8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00555db2cc544f0496e7a191f4805160', 'request_id': 'req-eec77bf6-ee6e-4b6f-ab3e-2ea4dbf11db8'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:58.641656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-eec77bf6-ee6e-4b6f-ab3e-2ea4dbf11db8 admin admin] The token cache is being invalidate because project 00555db2cc544f0496e7a191f4805160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:15:58.642805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-eec77bf6-ee6e-4b6f-ab3e-2ea4dbf11db8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:58.643583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-eec77bf6-ee6e-4b6f-ab3e-2ea4dbf11db8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:15:58.645263 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1064/2119] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:15:58 2022] DELETE /identity/v3/projects/00555db2cc544f0496e7a191f4805160 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:58.650690 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd364ed-06d6-4c70-be4c-9c549d336aa6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:58.651233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd364ed-06d6-4c70-be4c-9c549d336aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:58.651378 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd364ed-06d6-4c70-be4c-9c549d336aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:58.685341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-edd364ed-06d6-4c70-be4c-9c549d336aa6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:58.687421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-edd364ed-06d6-4c70-be4c-9c549d336aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:58.780813 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1056/2120] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:58.782815 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7586213-9c6f-4028-86a0-13c4f7b08171 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:58.783114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7586213-9c6f-4028-86a0-13c4f7b08171 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:58.783411 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7586213-9c6f-4028-86a0-13c4f7b08171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:58.805439 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e7586213-9c6f-4028-86a0-13c4f7b08171 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:58.806720 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e7586213-9c6f-4028-86a0-13c4f7b08171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:58.815542 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1065/2121] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:58.820031 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd364ed-06d6-4c70-be4c-9c549d336aa6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:58.830510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd364ed-06d6-4c70-be4c-9c549d336aa6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:58.835438 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c19cab-4324-421b-93ac-39fc889b0e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:58.836263 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c19cab-4324-421b-93ac-39fc889b0e29 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:58.836547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c19cab-4324-421b-93ac-39fc889b0e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:58.836812 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c19cab-4324-421b-93ac-39fc889b0e29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:58.837462 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c19cab-4324-421b-93ac-39fc889b0e29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:15:58.837763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c19cab-4324-421b-93ac-39fc889b0e29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:58.839517 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c19cab-4324-421b-93ac-39fc889b0e29 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:58.853640 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1057/2122] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:15:58.858462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7586213-9c6f-4028-86a0-13c4f7b08171 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:58.869751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7586213-9c6f-4028-86a0-13c4f7b08171 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:58.875554 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd7a3a-72d7-4062-b5a3-3c015aaf7af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:58.876236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd7a3a-72d7-4062-b5a3-3c015aaf7af0 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:58.877049 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd7a3a-72d7-4062-b5a3-3c015aaf7af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:58.877049 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd7a3a-72d7-4062-b5a3-3c015aaf7af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:58.877570 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd7a3a-72d7-4062-b5a3-3c015aaf7af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:58.880079 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd7a3a-72d7-4062-b5a3-3c015aaf7af0 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:58.912549 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1066/2123] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:58 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:58.916988 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c19cab-4324-421b-93ac-39fc889b0e29 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:58.928371 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c19cab-4324-421b-93ac-39fc889b0e29 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:58.997617 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f4e68-7fd9-4a75-9456-ebc2cf369087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:58.998301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f4e68-7fd9-4a75-9456-ebc2cf369087 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:58.998583 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f4e68-7fd9-4a75-9456-ebc2cf369087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:58.998888 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f4e68-7fd9-4a75-9456-ebc2cf369087 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:58.999544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f4e68-7fd9-4a75-9456-ebc2cf369087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:59.002362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f4e68-7fd9-4a75-9456-ebc2cf369087 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:59.041878 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:59.057400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1058/2124] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:58 2022] POST /identity/v3/users => generated 289 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:59.062801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd7a3a-72d7-4062-b5a3-3c015aaf7af0 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:59.082225 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd7a3a-72d7-4062-b5a3-3c015aaf7af0 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:59.092282 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bceb4-1ca4-492f-b79f-db51c23e2d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:59.092917 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bceb4-1ca4-492f-b79f-db51c23e2d00 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:59.093197 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bceb4-1ca4-492f-b79f-db51c23e2d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:59.093469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bceb4-1ca4-492f-b79f-db51c23e2d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:59.094198 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76bceb4-1ca4-492f-b79f-db51c23e2d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:59.097441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76bceb4-1ca4-492f-b79f-db51c23e2d00 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:59.122703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1067/2125] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:15:59.125307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f4e68-7fd9-4a75-9456-ebc2cf369087 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:59.143377 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f4e68-7fd9-4a75-9456-ebc2cf369087 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:59.148301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd59fe-6032-45f9-8b60-351a20a020e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:59.148930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd59fe-6032-45f9-8b60-351a20a020e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:59.149208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd59fe-6032-45f9-8b60-351a20a020e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:59.149535 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd59fe-6032-45f9-8b60-351a20a020e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:59.150187 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd59fe-6032-45f9-8b60-351a20a020e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:59.151865 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd59fe-6032-45f9-8b60-351a20a020e0 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:59.164777 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1059/2126] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:59.169333 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bceb4-1ca4-492f-b79f-db51c23e2d00 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:59.181838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bceb4-1ca4-492f-b79f-db51c23e2d00 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:59.188654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e3004-f477-4b8d-ae04-01a4c2571427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:59.189415 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e3004-f477-4b8d-ae04-01a4c2571427 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:59.189807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e3004-f477-4b8d-ae04-01a4c2571427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:59.190150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e3004-f477-4b8d-ae04-01a4c2571427 admin admin] PATH_INFO: `/v3/projects/5e5fe916f37042edb86eb9828b5a5f80/users/11c43bcf266c495587113304eecdf3eb/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:59.192869 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:15:59.211862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e3004-f477-4b8d-ae04-01a4c2571427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e5fe916f37042edb86eb9828b5a5f80, user_id=11c43bcf266c495587113304eecdf3eb, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:59.215535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e3004-f477-4b8d-ae04-01a4c2571427 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:59.232757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1068/2127] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:59 2022] PUT /identity/v3/projects/5e5fe916f37042edb86eb9828b5a5f80/users/11c43bcf266c495587113304eecdf3eb/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:15:59.244942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb00f3-c4b2-4f9b-aa73-c3041df5250b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:59.245455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb00f3-c4b2-4f9b-aa73-c3041df5250b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:59.245995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb00f3-c4b2-4f9b-aa73-c3041df5250b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:59.295642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-fbbb00f3-c4b2-4f9b-aa73-c3041df5250b None None] MFA Rules not processed for user `11c43bcf266c495587113304eecdf3eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:59.297696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fbbb00f3-c4b2-4f9b-aa73-c3041df5250b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:59.383905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1060/2128] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:15:59.388530 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e3004-f477-4b8d-ae04-01a4c2571427 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:59.481383 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e3004-f477-4b8d-ae04-01a4c2571427 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:59.488661 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d332512-52fa-497a-83c1-cb349743a78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:59.489289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d332512-52fa-497a-83c1-cb349743a78e admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:59.489628 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d332512-52fa-497a-83c1-cb349743a78e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:59.489917 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d332512-52fa-497a-83c1-cb349743a78e admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:59.490571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d332512-52fa-497a-83c1-cb349743a78e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:59.492564 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d332512-52fa-497a-83c1-cb349743a78e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:59.534878 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1069/2129] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:15:59 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:15:59.539095 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb00f3-c4b2-4f9b-aa73-c3041df5250b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:59.550548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb00f3-c4b2-4f9b-aa73-c3041df5250b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:59.626674 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231aa8f-7cd2-400a-8e08-89d02b03d597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:59.627304 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231aa8f-7cd2-400a-8e08-89d02b03d597 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:59.627575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231aa8f-7cd2-400a-8e08-89d02b03d597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:59.627849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231aa8f-7cd2-400a-8e08-89d02b03d597 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:59.628493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231aa8f-7cd2-400a-8e08-89d02b03d597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:59.630419 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231aa8f-7cd2-400a-8e08-89d02b03d597 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:59.662749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1061/2130] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:15:59 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:15:59.663719 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d332512-52fa-497a-83c1-cb349743a78e admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:59.681237 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d332512-52fa-497a-83c1-cb349743a78e admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:59.686510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3d7a5-263e-46b8-95ce-edf3f9608041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:59.687268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3d7a5-263e-46b8-95ce-edf3f9608041 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:59.687552 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3d7a5-263e-46b8-95ce-edf3f9608041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:59.687828 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3d7a5-263e-46b8-95ce-edf3f9608041 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:59.690047 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3d7a5-263e-46b8-95ce-edf3f9608041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:59.691720 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3d7a5-263e-46b8-95ce-edf3f9608041 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:59.703699 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1070/2131] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:59.709124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231aa8f-7cd2-400a-8e08-89d02b03d597 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:59.721432 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231aa8f-7cd2-400a-8e08-89d02b03d597 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:59.726604 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6af10-a8be-46d6-890e-e36bcfaead41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:59.727620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6af10-a8be-46d6-890e-e36bcfaead41 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:59.728017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6af10-a8be-46d6-890e-e36bcfaead41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:59.728335 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6af10-a8be-46d6-890e-e36bcfaead41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:59.729010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6af10-a8be-46d6-890e-e36bcfaead41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:59.730741 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6af10-a8be-46d6-890e-e36bcfaead41 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:59.742393 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1062/2132] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:15:59.747361 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3d7a5-263e-46b8-95ce-edf3f9608041 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:15:59.758938 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3d7a5-263e-46b8-95ce-edf3f9608041 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:15:59.763832 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad78d93-07a1-4b33-a6b6-9ba8a3ecb235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:15:59.765260 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad78d93-07a1-4b33-a6b6-9ba8a3ecb235 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:59.765550 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad78d93-07a1-4b33-a6b6-9ba8a3ecb235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:59.765822 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad78d93-07a1-4b33-a6b6-9ba8a3ecb235 admin admin] PATH_INFO: `/v3/projects/864ec35977fe431397d58991e4f487ce/users/5eba1683ec054ad0b792dfe0e876d327/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:59.785820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad78d93-07a1-4b33-a6b6-9ba8a3ecb235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=864ec35977fe431397d58991e4f487ce, user_id=5eba1683ec054ad0b792dfe0e876d327, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:15:59.789742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad78d93-07a1-4b33-a6b6-9ba8a3ecb235 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:15:59.814432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1071/2133] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:15:59 2022] PUT /identity/v3/projects/864ec35977fe431397d58991e4f487ce/users/5eba1683ec054ad0b792dfe0e876d327/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:15:59.833974 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05c008-f00d-45bf-a61e-4fb1b43f80f0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:15:59.834454 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05c008-f00d-45bf-a61e-4fb1b43f80f0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:15:59.835074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05c008-f00d-45bf-a61e-4fb1b43f80f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:15:59.888643 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-bc05c008-f00d-45bf-a61e-4fb1b43f80f0 None None] MFA Rules not processed for user `5eba1683ec054ad0b792dfe0e876d327`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:15:59.889892 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bc05c008-f00d-45bf-a61e-4fb1b43f80f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:15:59.963877 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:00.008671 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1063/2134] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:15:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:00.023365 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad78d93-07a1-4b33-a6b6-9ba8a3ecb235 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:00.132294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad78d93-07a1-4b33-a6b6-9ba8a3ecb235 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:00.142444 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f91377-7e63-4d7d-9316-ed4362f18e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:00.143707 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f91377-7e63-4d7d-9316-ed4362f18e12 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:00.144385 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f91377-7e63-4d7d-9316-ed4362f18e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:00.144926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f91377-7e63-4d7d-9316-ed4362f18e12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:00.146334 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f91377-7e63-4d7d-9316-ed4362f18e12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:00.154168 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f91377-7e63-4d7d-9316-ed4362f18e12 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:00.240561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1072/2135] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:00 2022] POST /identity/v3/projects => generated 375 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:00.246955 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05c008-f00d-45bf-a61e-4fb1b43f80f0 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:00.277204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05c008-f00d-45bf-a61e-4fb1b43f80f0 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:00.393741 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd8aed-8676-45bb-badf-41110e6c5ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:00.394284 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd8aed-8676-45bb-badf-41110e6c5ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:00.394564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd8aed-8676-45bb-badf-41110e6c5ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:00.396421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd8aed-8676-45bb-badf-41110e6c5ce6 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:00.397093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd8aed-8676-45bb-badf-41110e6c5ce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:00.399455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd8aed-8676-45bb-badf-41110e6c5ce6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:00.427404 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1064/2136] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:00 2022] POST /identity/v3/users => generated 294 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:00.431805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f91377-7e63-4d7d-9316-ed4362f18e12 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:00.452813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f91377-7e63-4d7d-9316-ed4362f18e12 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:00.459426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012060c5-e961-4124-bb91-1c23880a8d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:00.460384 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012060c5-e961-4124-bb91-1c23880a8d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:00.461081 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012060c5-e961-4124-bb91-1c23880a8d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:00.461694 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012060c5-e961-4124-bb91-1c23880a8d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:00.462556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012060c5-e961-4124-bb91-1c23880a8d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:00.466126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012060c5-e961-4124-bb91-1c23880a8d39 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:00.480300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1073/2137] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:16:00.484551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd8aed-8676-45bb-badf-41110e6c5ce6 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:00.498973 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd8aed-8676-45bb-badf-41110e6c5ce6 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:00.505201 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89749854-32a2-443e-84b4-1333f536f798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:00.506062 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89749854-32a2-443e-84b4-1333f536f798 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:00.506413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89749854-32a2-443e-84b4-1333f536f798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:00.506850 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89749854-32a2-443e-84b4-1333f536f798 admin admin] PATH_INFO: `/v3/projects/ccd3bc56f12b41d09806e8bc10b04ca5/users/8a13adbc63c74be6871c638f05c23395/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:00.537745 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89749854-32a2-443e-84b4-1333f536f798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccd3bc56f12b41d09806e8bc10b04ca5, user_id=8a13adbc63c74be6871c638f05c23395, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:00.540846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89749854-32a2-443e-84b4-1333f536f798 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:00.563468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1065/2138] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:00 2022] PUT /identity/v3/projects/ccd3bc56f12b41d09806e8bc10b04ca5/users/8a13adbc63c74be6871c638f05c23395/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:16:00.567681 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012060c5-e961-4124-bb91-1c23880a8d39 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:00.582165 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012060c5-e961-4124-bb91-1c23880a8d39 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:00.589639 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:00.659861 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e3fac-affe-4d66-a359-e6fb5ca6d14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:00.660487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e3fac-affe-4d66-a359-e6fb5ca6d14d admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:00.660763 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e3fac-affe-4d66-a359-e6fb5ca6d14d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:00.661027 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e3fac-affe-4d66-a359-e6fb5ca6d14d admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:00.661688 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e3fac-affe-4d66-a359-e6fb5ca6d14d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:00.663334 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e3fac-affe-4d66-a359-e6fb5ca6d14d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:00.676220 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1074/2139] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:16:00.680514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89749854-32a2-443e-84b4-1333f536f798 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:00.693448 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89749854-32a2-443e-84b4-1333f536f798 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:00.698459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e9060-049c-46df-93a6-a90e17dbe82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:00.699199 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e9060-049c-46df-93a6-a90e17dbe82a admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:00.699498 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e9060-049c-46df-93a6-a90e17dbe82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:00.699773 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e9060-049c-46df-93a6-a90e17dbe82a admin admin] PATH_INFO: `/v3/projects/ccd3bc56f12b41d09806e8bc10b04ca5/users/8a13adbc63c74be6871c638f05c23395/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:00.701811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e9060-049c-46df-93a6-a90e17dbe82a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccd3bc56f12b41d09806e8bc10b04ca5, user_id=8a13adbc63c74be6871c638f05c23395, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:00.705095 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e9060-049c-46df-93a6-a90e17dbe82a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:00.724304 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1066/2140] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:00 2022] PUT /identity/v3/projects/ccd3bc56f12b41d09806e8bc10b04ca5/users/8a13adbc63c74be6871c638f05c23395/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:00.734726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab33555-be79-4be1-a0d1-8d9fce234362 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:00.735022 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab33555-be79-4be1-a0d1-8d9fce234362 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:00.735337 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab33555-be79-4be1-a0d1-8d9fce234362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:00.767928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6ab33555-be79-4be1-a0d1-8d9fce234362 None None] MFA Rules not processed for user `8a13adbc63c74be6871c638f05c23395`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:00.769148 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6ab33555-be79-4be1-a0d1-8d9fce234362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:00.868457 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1075/2141] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:00.887147 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e9060-049c-46df-93a6-a90e17dbe82a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:00.889809 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-416e9060-049c-46df-93a6-a90e17dbe82a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:00.959829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e9060-049c-46df-93a6-a90e17dbe82a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:00.964218 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cf0d5-076e-417b-831b-10d52b765e7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:00.964840 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cf0d5-076e-417b-831b-10d52b765e7d service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:00.965122 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cf0d5-076e-417b-831b-10d52b765e7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:00.965400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cf0d5-076e-417b-831b-10d52b765e7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:00.975508 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cf0d5-076e-417b-831b-10d52b765e7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:00.977047 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cf0d5-076e-417b-831b-10d52b765e7d service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:00.989991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1067/2142] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:01.060681 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab33555-be79-4be1-a0d1-8d9fce234362 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:01.072604 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab33555-be79-4be1-a0d1-8d9fce234362 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:01.076240 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec16472-ca08-4444-a777-43a6bc2885bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:01.076840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec16472-ca08-4444-a777-43a6bc2885bf service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:01.077116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec16472-ca08-4444-a777-43a6bc2885bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:01.077436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec16472-ca08-4444-a777-43a6bc2885bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:01.157494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec16472-ca08-4444-a777-43a6bc2885bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:01.159115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec16472-ca08-4444-a777-43a6bc2885bf service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:01.175743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1076/2143] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:01.264527 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cf0d5-076e-417b-831b-10d52b765e7d service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:01.267599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9f9cf0d5-076e-417b-831b-10d52b765e7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:01.381412 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:01.418455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cf0d5-076e-417b-831b-10d52b765e7d service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:01.424936 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed1aba-1454-4e76-b893-6213d92c2698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:01.425780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed1aba-1454-4e76-b893-6213d92c2698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:01.426074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed1aba-1454-4e76-b893-6213d92c2698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:01.426339 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed1aba-1454-4e76-b893-6213d92c2698 admin admin] PATH_INFO: `/v3/users/96c766aa092c4af185687decfb0db0f0` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:01.427617 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed1aba-1454-4e76-b893-6213d92c2698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96c766aa092c4af185687decfb0db0f0)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:01.429622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed1aba-1454-4e76-b893-6213d92c2698 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:01.502648 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-5fed1aba-1454-4e76-b893-6213d92c2698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96c766aa092c4af185687decfb0db0f0'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.514800 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-5fed1aba-1454-4e76-b893-6213d92c2698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96c766aa092c4af185687decfb0db0f0'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.539862 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-5fed1aba-1454-4e76-b893-6213d92c2698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96c766aa092c4af185687decfb0db0f0'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.540324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-5fed1aba-1454-4e76-b893-6213d92c2698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96c766aa092c4af185687decfb0db0f0'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.543138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-5fed1aba-1454-4e76-b893-6213d92c2698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96c766aa092c4af185687decfb0db0f0'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:01.562034 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1068/2144] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:01 2022] DELETE /identity/v3/users/96c766aa092c4af185687decfb0db0f0 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:01.565733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29127dff-3713-4bc7-bba1-1a65accd48c4 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:01.566095 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29127dff-3713-4bc7-bba1-1a65accd48c4 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:01.566447 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29127dff-3713-4bc7-bba1-1a65accd48c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:01.587825 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-29127dff-3713-4bc7-bba1-1a65accd48c4 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:01.589250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-29127dff-3713-4bc7-bba1-1a65accd48c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:01.616045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:01.692363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1077/2145] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:01.705761 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed1aba-1454-4e76-b893-6213d92c2698 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:01.707476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5fed1aba-1454-4e76-b893-6213d92c2698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:01.800383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed1aba-1454-4e76-b893-6213d92c2698 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:01.804541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8f36f-336c-40cd-902c-7cfb96f7679a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:01.805170 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8f36f-336c-40cd-902c-7cfb96f7679a service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:01.805447 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8f36f-336c-40cd-902c-7cfb96f7679a service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:01.805824 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8f36f-336c-40cd-902c-7cfb96f7679a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:01.817947 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8f36f-336c-40cd-902c-7cfb96f7679a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:01.819383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8f36f-336c-40cd-902c-7cfb96f7679a service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:01.831493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1069/2146] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:02.196455 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29127dff-3713-4bc7-bba1-1a65accd48c4 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:02.197834 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-29127dff-3713-4bc7-bba1-1a65accd48c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:02.211494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29127dff-3713-4bc7-bba1-1a65accd48c4 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:02.216767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5372a1-0ab7-4dd5-91e6-71e985e5c560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:02.217366 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5372a1-0ab7-4dd5-91e6-71e985e5c560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:02.217651 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5372a1-0ab7-4dd5-91e6-71e985e5c560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:02.217927 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5372a1-0ab7-4dd5-91e6-71e985e5c560 admin admin] PATH_INFO: `/v3/projects/754375a274c54a84a7db99c33218c97a` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:02.226717 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5372a1-0ab7-4dd5-91e6-71e985e5c560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=754375a274c54a84a7db99c33218c97a)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:02.229063 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5372a1-0ab7-4dd5-91e6-71e985e5c560 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:02.293266 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-5f5372a1-0ab7-4dd5-91e6-71e985e5c560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '754375a274c54a84a7db99c33218c97a', 'request_id': 'req-5f5372a1-0ab7-4dd5-91e6-71e985e5c560'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:02.300460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-5f5372a1-0ab7-4dd5-91e6-71e985e5c560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '754375a274c54a84a7db99c33218c97a', 'request_id': 'req-5f5372a1-0ab7-4dd5-91e6-71e985e5c560'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:02.306935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-5f5372a1-0ab7-4dd5-91e6-71e985e5c560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '754375a274c54a84a7db99c33218c97a', 'request_id': 'req-5f5372a1-0ab7-4dd5-91e6-71e985e5c560'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:02.322990 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-5f5372a1-0ab7-4dd5-91e6-71e985e5c560 admin admin] The token cache is being invalidate because project 754375a274c54a84a7db99c33218c97a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:02.323327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-5f5372a1-0ab7-4dd5-91e6-71e985e5c560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:02.324087 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-5f5372a1-0ab7-4dd5-91e6-71e985e5c560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:02.325808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1078/2147] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:02 2022] DELETE /identity/v3/projects/754375a274c54a84a7db99c33218c97a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:02.333096 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8f36f-336c-40cd-902c-7cfb96f7679a service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:02.350018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8f36f-336c-40cd-902c-7cfb96f7679a service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:02.442090 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7c19a-3c8f-413a-b767-5fe2d615a547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:02.443148 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7c19a-3c8f-413a-b767-5fe2d615a547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:02.443611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7c19a-3c8f-413a-b767-5fe2d615a547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:02.444090 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7c19a-3c8f-413a-b767-5fe2d615a547 admin admin] PATH_INFO: `/v3/users/7da0358d74db43fbb541c80ffe9b8f8f` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:02.445609 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7c19a-3c8f-413a-b767-5fe2d615a547 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7da0358d74db43fbb541c80ffe9b8f8f)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:02.448340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7c19a-3c8f-413a-b767-5fe2d615a547 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:02.451928 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:02.513511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4eb7c19a-3c8f-413a-b767-5fe2d615a547 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7da0358d74db43fbb541c80ffe9b8f8f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:02.522154 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4eb7c19a-3c8f-413a-b767-5fe2d615a547 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7da0358d74db43fbb541c80ffe9b8f8f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:02.535780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4eb7c19a-3c8f-413a-b767-5fe2d615a547 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7da0358d74db43fbb541c80ffe9b8f8f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:02.536529 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4eb7c19a-3c8f-413a-b767-5fe2d615a547 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7da0358d74db43fbb541c80ffe9b8f8f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:02.537335 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4eb7c19a-3c8f-413a-b767-5fe2d615a547 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7da0358d74db43fbb541c80ffe9b8f8f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:02.554157 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1070/2148] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:02 2022] DELETE /identity/v3/users/7da0358d74db43fbb541c80ffe9b8f8f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:02.973756 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5372a1-0ab7-4dd5-91e6-71e985e5c560 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:02.977348 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5f5372a1-0ab7-4dd5-91e6-71e985e5c560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:02.998905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5372a1-0ab7-4dd5-91e6-71e985e5c560 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:03.078650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cdcf11-89ae-4ae1-9342-6e3171a35868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:03.079427 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdcf11-89ae-4ae1-9342-6e3171a35868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.079701 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdcf11-89ae-4ae1-9342-6e3171a35868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.080005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdcf11-89ae-4ae1-9342-6e3171a35868 admin admin] PATH_INFO: `/v3/projects/5afa858b8dfa4e9095e28736b579b4ef` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.081253 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cdcf11-89ae-4ae1-9342-6e3171a35868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5afa858b8dfa4e9095e28736b579b4ef)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:03.083336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cdcf11-89ae-4ae1-9342-6e3171a35868 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:03.151638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-84cdcf11-89ae-4ae1-9342-6e3171a35868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5afa858b8dfa4e9095e28736b579b4ef', 'request_id': 'req-84cdcf11-89ae-4ae1-9342-6e3171a35868'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:03.155030 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:03.161364 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-84cdcf11-89ae-4ae1-9342-6e3171a35868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5afa858b8dfa4e9095e28736b579b4ef', 'request_id': 'req-84cdcf11-89ae-4ae1-9342-6e3171a35868'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:03.170292 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-84cdcf11-89ae-4ae1-9342-6e3171a35868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5afa858b8dfa4e9095e28736b579b4ef', 'request_id': 'req-84cdcf11-89ae-4ae1-9342-6e3171a35868'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:03.185557 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-84cdcf11-89ae-4ae1-9342-6e3171a35868 admin admin] The token cache is being invalidate because project 5afa858b8dfa4e9095e28736b579b4ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:03.185852 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-84cdcf11-89ae-4ae1-9342-6e3171a35868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:03.186523 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-84cdcf11-89ae-4ae1-9342-6e3171a35868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:03.188360 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1079/2149] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:02 2022] DELETE /identity/v3/projects/5afa858b8dfa4e9095e28736b579b4ef => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:03.193995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965a83b-48cd-49b5-9227-ca44b8aec225 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.194508 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965a83b-48cd-49b5-9227-ca44b8aec225 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.194621 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965a83b-48cd-49b5-9227-ca44b8aec225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.216553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0965a83b-48cd-49b5-9227-ca44b8aec225 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:03.217972 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0965a83b-48cd-49b5-9227-ca44b8aec225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:03.309137 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1071/2150] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:03.318836 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6f625-5571-45da-890f-2697033abef8 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.319137 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6f625-5571-45da-890f-2697033abef8 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.319624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6f625-5571-45da-890f-2697033abef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.351106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d0a6f625-5571-45da-890f-2697033abef8 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:03.352878 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d0a6f625-5571-45da-890f-2697033abef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:03.365933 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1080/2151] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:16:03.372214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965a83b-48cd-49b5-9227-ca44b8aec225 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:03.387555 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965a83b-48cd-49b5-9227-ca44b8aec225 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:03.395604 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50604d-9031-4385-a239-a2aac49c7d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:03.396650 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50604d-9031-4385-a239-a2aac49c7d5f admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.397119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50604d-9031-4385-a239-a2aac49c7d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.397582 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50604d-9031-4385-a239-a2aac49c7d5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.398863 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50604d-9031-4385-a239-a2aac49c7d5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:16:03.399292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50604d-9031-4385-a239-a2aac49c7d5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:03.401305 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50604d-9031-4385-a239-a2aac49c7d5f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:03.421531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1072/2152] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:16:03.428162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6f625-5571-45da-890f-2697033abef8 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:03.442728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6f625-5571-45da-890f-2697033abef8 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:03.456946 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045bea8-748b-472e-9512-04f756b7bfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:03.456946 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045bea8-748b-472e-9512-04f756b7bfe8 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.456946 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045bea8-748b-472e-9512-04f756b7bfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.456946 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045bea8-748b-472e-9512-04f756b7bfe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.457591 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045bea8-748b-472e-9512-04f756b7bfe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:03.459461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045bea8-748b-472e-9512-04f756b7bfe8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:03.508035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1081/2153] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:03 2022] POST /identity/v3/projects => generated 401 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:16:03.512746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50604d-9031-4385-a239-a2aac49c7d5f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:03.524113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50604d-9031-4385-a239-a2aac49c7d5f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:03.586588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663e5cc-be21-4e59-9f6d-28480704a96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:03.587273 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663e5cc-be21-4e59-9f6d-28480704a96d admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.587602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663e5cc-be21-4e59-9f6d-28480704a96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.587911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663e5cc-be21-4e59-9f6d-28480704a96d admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.588568 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a663e5cc-be21-4e59-9f6d-28480704a96d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:03.590601 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a663e5cc-be21-4e59-9f6d-28480704a96d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:03.609470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:03.620029 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1073/2154] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:03 2022] POST /identity/v3/users => generated 301 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:03.625146 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045bea8-748b-472e-9512-04f756b7bfe8 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:03.636913 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045bea8-748b-472e-9512-04f756b7bfe8 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:03.642251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f020891-38cb-44b9-a256-5207fce68a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:03.642902 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f020891-38cb-44b9-a256-5207fce68a62 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.643182 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f020891-38cb-44b9-a256-5207fce68a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.643454 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f020891-38cb-44b9-a256-5207fce68a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.644104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f020891-38cb-44b9-a256-5207fce68a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:03.645712 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f020891-38cb-44b9-a256-5207fce68a62 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:03.658508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1082/2155] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:03.663794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663e5cc-be21-4e59-9f6d-28480704a96d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:03.675396 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663e5cc-be21-4e59-9f6d-28480704a96d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:03.680384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eef8c4-918f-4264-a872-0d5ca3f7ff51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:03.681049 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eef8c4-918f-4264-a872-0d5ca3f7ff51 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.681471 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eef8c4-918f-4264-a872-0d5ca3f7ff51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.681758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eef8c4-918f-4264-a872-0d5ca3f7ff51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.682417 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eef8c4-918f-4264-a872-0d5ca3f7ff51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:03.684233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eef8c4-918f-4264-a872-0d5ca3f7ff51 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:03.697682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1074/2156] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:03.703741 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f020891-38cb-44b9-a256-5207fce68a62 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:03.715745 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f020891-38cb-44b9-a256-5207fce68a62 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:03.721929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca241cf1-3627-4d62-a471-ff197c1efc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:03.722710 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca241cf1-3627-4d62-a471-ff197c1efc0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.723025 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca241cf1-3627-4d62-a471-ff197c1efc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.723329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca241cf1-3627-4d62-a471-ff197c1efc0b admin admin] PATH_INFO: `/v3/projects/44b046e5296a4438804420c65973da53/users/8ccfd694989249a78451b1546ac77105/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.740651 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca241cf1-3627-4d62-a471-ff197c1efc0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b046e5296a4438804420c65973da53, user_id=8ccfd694989249a78451b1546ac77105, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:03.743410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca241cf1-3627-4d62-a471-ff197c1efc0b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:03.759671 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1083/2157] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:03 2022] PUT /identity/v3/projects/44b046e5296a4438804420c65973da53/users/8ccfd694989249a78451b1546ac77105/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:03.772371 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19718931-9494-4291-b705-b1a1b9224331 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.772775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19718931-9494-4291-b705-b1a1b9224331 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.773056 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19718931-9494-4291-b705-b1a1b9224331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.805773 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-19718931-9494-4291-b705-b1a1b9224331 None None] MFA Rules not processed for user `8ccfd694989249a78451b1546ac77105`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:03.807087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-19718931-9494-4291-b705-b1a1b9224331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:03.888873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1075/2158] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:03 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:03.890131 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca241cf1-3627-4d62-a471-ff197c1efc0b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:03.961552 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca241cf1-3627-4d62-a471-ff197c1efc0b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:03.966975 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7731b-69ef-43a1-9df7-4fe85226672c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:03.967583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7731b-69ef-43a1-9df7-4fe85226672c admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:03.967861 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7731b-69ef-43a1-9df7-4fe85226672c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:03.968234 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7731b-69ef-43a1-9df7-4fe85226672c admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:03.968902 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7731b-69ef-43a1-9df7-4fe85226672c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:03.971628 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7731b-69ef-43a1-9df7-4fe85226672c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:04.004492 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1084/2159] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:03 2022] POST /identity/v3/projects => generated 401 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:04.011246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19718931-9494-4291-b705-b1a1b9224331 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:04.024846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19718931-9494-4291-b705-b1a1b9224331 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:04.094044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21978db4-f8db-43d8-a098-91b97be1a597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:04.094892 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21978db4-f8db-43d8-a098-91b97be1a597 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:04.095368 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21978db4-f8db-43d8-a098-91b97be1a597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:04.095634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21978db4-f8db-43d8-a098-91b97be1a597 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.096410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21978db4-f8db-43d8-a098-91b97be1a597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:04.098373 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21978db4-f8db-43d8-a098-91b97be1a597 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:04.129039 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1076/2160] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:04 2022] POST /identity/v3/users => generated 307 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:04.134760 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7731b-69ef-43a1-9df7-4fe85226672c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:04.147274 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7731b-69ef-43a1-9df7-4fe85226672c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:04.154503 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b8a75-5cda-4c59-a044-a1b838066398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:04.155191 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b8a75-5cda-4c59-a044-a1b838066398 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:04.156121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b8a75-5cda-4c59-a044-a1b838066398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:04.156483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b8a75-5cda-4c59-a044-a1b838066398 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.160365 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b8a75-5cda-4c59-a044-a1b838066398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:04.163819 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b8a75-5cda-4c59-a044-a1b838066398 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:04.177591 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1085/2161] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:16:04.181726 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21978db4-f8db-43d8-a098-91b97be1a597 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:04.193791 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21978db4-f8db-43d8-a098-91b97be1a597 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:04.200638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c12f8d-4eb3-4d42-96f8-11b13bfd4e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:04.201370 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c12f8d-4eb3-4d42-96f8-11b13bfd4e1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:04.201745 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c12f8d-4eb3-4d42-96f8-11b13bfd4e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:04.202224 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c12f8d-4eb3-4d42-96f8-11b13bfd4e1b admin admin] PATH_INFO: `/v3/projects/f194ccf713ec49e1bf1b98345afb8650/users/8b28d478ad514d2292b141649dfe7227/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.224189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c12f8d-4eb3-4d42-96f8-11b13bfd4e1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f194ccf713ec49e1bf1b98345afb8650, user_id=8b28d478ad514d2292b141649dfe7227, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:04.226969 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c12f8d-4eb3-4d42-96f8-11b13bfd4e1b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:04.250426 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1077/2162] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:04 2022] PUT /identity/v3/projects/f194ccf713ec49e1bf1b98345afb8650/users/8b28d478ad514d2292b141649dfe7227/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:04.257006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b8a75-5cda-4c59-a044-a1b838066398 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:04.268458 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b8a75-5cda-4c59-a044-a1b838066398 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:04.325459 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:04.339978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8703b71d-282d-4dbd-b0fb-b243b9bf63cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:04.340692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703b71d-282d-4dbd-b0fb-b243b9bf63cd admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:04.340972 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703b71d-282d-4dbd-b0fb-b243b9bf63cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:04.341921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703b71d-282d-4dbd-b0fb-b243b9bf63cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.342622 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8703b71d-282d-4dbd-b0fb-b243b9bf63cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:04.344611 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8703b71d-282d-4dbd-b0fb-b243b9bf63cd admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:04.357772 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1086/2163] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:04.364044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c12f8d-4eb3-4d42-96f8-11b13bfd4e1b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:04.376003 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c12f8d-4eb3-4d42-96f8-11b13bfd4e1b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:04.381322 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04054f80-c4e8-4b5f-b638-114ec0fe6b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:04.382001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04054f80-c4e8-4b5f-b638-114ec0fe6b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:04.382286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04054f80-c4e8-4b5f-b638-114ec0fe6b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:04.382581 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04054f80-c4e8-4b5f-b638-114ec0fe6b43 admin admin] PATH_INFO: `/v3/projects/f194ccf713ec49e1bf1b98345afb8650/users/8b28d478ad514d2292b141649dfe7227/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.384595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04054f80-c4e8-4b5f-b638-114ec0fe6b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f194ccf713ec49e1bf1b98345afb8650, user_id=8b28d478ad514d2292b141649dfe7227, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:04.387556 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04054f80-c4e8-4b5f-b638-114ec0fe6b43 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:04.399471 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1078/2164] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:04 2022] PUT /identity/v3/projects/f194ccf713ec49e1bf1b98345afb8650/users/8b28d478ad514d2292b141649dfe7227/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:04.409994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4b696-9907-48e4-8849-1d19d905bb5f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:04.410377 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4b696-9907-48e4-8849-1d19d905bb5f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:04.410787 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4b696-9907-48e4-8849-1d19d905bb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.446546 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-83a4b696-9907-48e4-8849-1d19d905bb5f None None] MFA Rules not processed for user `8b28d478ad514d2292b141649dfe7227`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:04.447813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-83a4b696-9907-48e4-8849-1d19d905bb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:04.543650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1087/2165] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:04 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:04.554731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04054f80-c4e8-4b5f-b638-114ec0fe6b43 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:04.561087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-04054f80-c4e8-4b5f-b638-114ec0fe6b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:04.630798 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:04.677651 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04054f80-c4e8-4b5f-b638-114ec0fe6b43 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:04.684858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6d955-9dd9-498e-82fe-cd8fdf0fc4c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:04.684858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6d955-9dd9-498e-82fe-cd8fdf0fc4c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:04.684858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6d955-9dd9-498e-82fe-cd8fdf0fc4c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:04.684858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6d955-9dd9-498e-82fe-cd8fdf0fc4c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.695909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6d955-9dd9-498e-82fe-cd8fdf0fc4c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:04.697619 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6d955-9dd9-498e-82fe-cd8fdf0fc4c0 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:04.709587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1079/2166] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:04 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:16:04.794172 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4b696-9907-48e4-8849-1d19d905bb5f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:04.805167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4b696-9907-48e4-8849-1d19d905bb5f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:04.809593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3fa9a-cdd8-4f55-8e55-7f051423c893 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:04.810265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3fa9a-cdd8-4f55-8e55-7f051423c893 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:04.810552 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3fa9a-cdd8-4f55-8e55-7f051423c893 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:04.810860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3fa9a-cdd8-4f55-8e55-7f051423c893 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:04.873926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3fa9a-cdd8-4f55-8e55-7f051423c893 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:04.875482 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3fa9a-cdd8-4f55-8e55-7f051423c893 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:04.886968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1088/2167] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:04 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:11.135561 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6d955-9dd9-498e-82fe-cd8fdf0fc4c0 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:11.137920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d8e6d955-9dd9-498e-82fe-cd8fdf0fc4c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:11.294532 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6d955-9dd9-498e-82fe-cd8fdf0fc4c0 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:11.303679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3225f-8024-4393-ba3a-9aa111a6d905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:11.304920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3225f-8024-4393-ba3a-9aa111a6d905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:11.305321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3225f-8024-4393-ba3a-9aa111a6d905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:11.305644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3225f-8024-4393-ba3a-9aa111a6d905 admin admin] PATH_INFO: `/v3/users/11c43bcf266c495587113304eecdf3eb` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:11.307807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3225f-8024-4393-ba3a-9aa111a6d905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11c43bcf266c495587113304eecdf3eb)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:11.311394 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3225f-8024-4393-ba3a-9aa111a6d905 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:11.409023 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8fd3225f-8024-4393-ba3a-9aa111a6d905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11c43bcf266c495587113304eecdf3eb'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:11.421793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8fd3225f-8024-4393-ba3a-9aa111a6d905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11c43bcf266c495587113304eecdf3eb'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:11.445621 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8fd3225f-8024-4393-ba3a-9aa111a6d905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11c43bcf266c495587113304eecdf3eb'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:11.446380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8fd3225f-8024-4393-ba3a-9aa111a6d905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11c43bcf266c495587113304eecdf3eb'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:11.447051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8fd3225f-8024-4393-ba3a-9aa111a6d905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11c43bcf266c495587113304eecdf3eb'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:11.465176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1080/2168] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:11 2022] DELETE /identity/v3/users/11c43bcf266c495587113304eecdf3eb => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:16:11.471243 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c53e25-4cd5-4552-be1b-5226556e9dc1 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:11.472044 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c53e25-4cd5-4552-be1b-5226556e9dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:11.472044 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c53e25-4cd5-4552-be1b-5226556e9dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:11.496063 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-23c53e25-4cd5-4552-be1b-5226556e9dc1 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:11.500129 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-23c53e25-4cd5-4552-be1b-5226556e9dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:11.606801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75f6a3-b153-4465-91ae-e4d8600a4449 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:11.607208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75f6a3-b153-4465-91ae-e4d8600a4449 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:11.607584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75f6a3-b153-4465-91ae-e4d8600a4449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:11.629468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1089/2169] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:11.631957 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-7d75f6a3-b153-4465-91ae-e4d8600a4449 None None] MFA Rules not processed for user `b24f1542cddf4fc9aa8003952e26cb96`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:11.633711 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7d75f6a3-b153-4465-91ae-e4d8600a4449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:11.640579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c53e25-4cd5-4552-be1b-5226556e9dc1 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:11.643749 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-23c53e25-4cd5-4552-be1b-5226556e9dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:11.663641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:11.702999 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:11.727180 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1081/2170] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:16:11.735378 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c53e25-4cd5-4552-be1b-5226556e9dc1 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:11.741766 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86be223-1340-44f4-988b-eda33ca2a0ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:11.743021 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86be223-1340-44f4-988b-eda33ca2a0ce service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:11.744426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86be223-1340-44f4-988b-eda33ca2a0ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:11.744426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86be223-1340-44f4-988b-eda33ca2a0ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:11.758160 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86be223-1340-44f4-988b-eda33ca2a0ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:11.760290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86be223-1340-44f4-988b-eda33ca2a0ce service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:11.775843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1090/2171] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:12.159369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75f6a3-b153-4465-91ae-e4d8600a4449 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:12.166552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7d75f6a3-b153-4465-91ae-e4d8600a4449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:12.183016 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75f6a3-b153-4465-91ae-e4d8600a4449 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:12.189091 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18619f0f-fff0-4341-800b-65011da60b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:12.189758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18619f0f-fff0-4341-800b-65011da60b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:12.190046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18619f0f-fff0-4341-800b-65011da60b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:12.190314 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18619f0f-fff0-4341-800b-65011da60b6b admin admin] PATH_INFO: `/v3/projects/5e5fe916f37042edb86eb9828b5a5f80` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:12.191645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18619f0f-fff0-4341-800b-65011da60b6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e5fe916f37042edb86eb9828b5a5f80)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:12.193657 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18619f0f-fff0-4341-800b-65011da60b6b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:12.270568 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-18619f0f-fff0-4341-800b-65011da60b6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e5fe916f37042edb86eb9828b5a5f80', 'request_id': 'req-18619f0f-fff0-4341-800b-65011da60b6b'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:12.280724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-18619f0f-fff0-4341-800b-65011da60b6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e5fe916f37042edb86eb9828b5a5f80', 'request_id': 'req-18619f0f-fff0-4341-800b-65011da60b6b'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:12.289070 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-18619f0f-fff0-4341-800b-65011da60b6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e5fe916f37042edb86eb9828b5a5f80', 'request_id': 'req-18619f0f-fff0-4341-800b-65011da60b6b'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:12.304873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-18619f0f-fff0-4341-800b-65011da60b6b admin admin] The token cache is being invalidate because project 5e5fe916f37042edb86eb9828b5a5f80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:12.305208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-18619f0f-fff0-4341-800b-65011da60b6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:12.306086 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-18619f0f-fff0-4341-800b-65011da60b6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:12.308402 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1082/2172] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:12 2022] DELETE /identity/v3/projects/5e5fe916f37042edb86eb9828b5a5f80 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:12.315042 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86be223-1340-44f4-988b-eda33ca2a0ce service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:12.345169 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86be223-1340-44f4-988b-eda33ca2a0ce service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:12.509146 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85092b-e927-4f81-8a4e-1e7cf1bac740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:12.509832 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85092b-e927-4f81-8a4e-1e7cf1bac740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:12.510493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85092b-e927-4f81-8a4e-1e7cf1bac740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:12.511200 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85092b-e927-4f81-8a4e-1e7cf1bac740 admin admin] PATH_INFO: `/v3/users/5eba1683ec054ad0b792dfe0e876d327` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:12.513020 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85092b-e927-4f81-8a4e-1e7cf1bac740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eba1683ec054ad0b792dfe0e876d327)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:12.515249 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85092b-e927-4f81-8a4e-1e7cf1bac740 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:12.614081 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8b85092b-e927-4f81-8a4e-1e7cf1bac740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eba1683ec054ad0b792dfe0e876d327'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:12.634699 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8b85092b-e927-4f81-8a4e-1e7cf1bac740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eba1683ec054ad0b792dfe0e876d327'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:12.659638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8b85092b-e927-4f81-8a4e-1e7cf1bac740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eba1683ec054ad0b792dfe0e876d327'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:12.661066 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8b85092b-e927-4f81-8a4e-1e7cf1bac740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eba1683ec054ad0b792dfe0e876d327'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:12.682566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8b85092b-e927-4f81-8a4e-1e7cf1bac740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eba1683ec054ad0b792dfe0e876d327'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:12.714715 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1091/2173] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:12 2022] DELETE /identity/v3/users/5eba1683ec054ad0b792dfe0e876d327 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:13.319397 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18619f0f-fff0-4341-800b-65011da60b6b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:13.321964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-18619f0f-fff0-4341-800b-65011da60b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:13.350465 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18619f0f-fff0-4341-800b-65011da60b6b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:13.456703 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadfaa4b-505e-42bf-af29-eabfa88791fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:13.457362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadfaa4b-505e-42bf-af29-eabfa88791fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:13.457642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadfaa4b-505e-42bf-af29-eabfa88791fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:13.457920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadfaa4b-505e-42bf-af29-eabfa88791fd admin admin] PATH_INFO: `/v3/projects/864ec35977fe431397d58991e4f487ce` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:13.459218 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadfaa4b-505e-42bf-af29-eabfa88791fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=864ec35977fe431397d58991e4f487ce)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:13.461449 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadfaa4b-505e-42bf-af29-eabfa88791fd admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:13.624426 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dadfaa4b-505e-42bf-af29-eabfa88791fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '864ec35977fe431397d58991e4f487ce', 'request_id': 'req-dadfaa4b-505e-42bf-af29-eabfa88791fd'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:13.644868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dadfaa4b-505e-42bf-af29-eabfa88791fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '864ec35977fe431397d58991e4f487ce', 'request_id': 'req-dadfaa4b-505e-42bf-af29-eabfa88791fd'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:13.659200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dadfaa4b-505e-42bf-af29-eabfa88791fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '864ec35977fe431397d58991e4f487ce', 'request_id': 'req-dadfaa4b-505e-42bf-af29-eabfa88791fd'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:13.689940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dadfaa4b-505e-42bf-af29-eabfa88791fd admin admin] The token cache is being invalidate because project 864ec35977fe431397d58991e4f487ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:13.690930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dadfaa4b-505e-42bf-af29-eabfa88791fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:13.692848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dadfaa4b-505e-42bf-af29-eabfa88791fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:13.698478 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1083/2174] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:13 2022] DELETE /identity/v3/projects/864ec35977fe431397d58991e4f487ce => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:13.703957 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85092b-e927-4f81-8a4e-1e7cf1bac740 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:13.710056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8b85092b-e927-4f81-8a4e-1e7cf1bac740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:13.728315 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85092b-e927-4f81-8a4e-1e7cf1bac740 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:13.775062 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:13.851423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3644e-3ee2-46cd-96df-c0d3bee79419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:13.852223 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3644e-3ee2-46cd-96df-c0d3bee79419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:13.852571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3644e-3ee2-46cd-96df-c0d3bee79419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:13.852923 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3644e-3ee2-46cd-96df-c0d3bee79419 admin admin] PATH_INFO: `/v3/users/8a13adbc63c74be6871c638f05c23395` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:13.854213 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3644e-3ee2-46cd-96df-c0d3bee79419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a13adbc63c74be6871c638f05c23395)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:13.856335 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3644e-3ee2-46cd-96df-c0d3bee79419 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:13.929507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ecd3644e-3ee2-46cd-96df-c0d3bee79419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a13adbc63c74be6871c638f05c23395'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:13.940708 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ecd3644e-3ee2-46cd-96df-c0d3bee79419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a13adbc63c74be6871c638f05c23395'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:13.956727 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ecd3644e-3ee2-46cd-96df-c0d3bee79419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a13adbc63c74be6871c638f05c23395'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:13.957650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ecd3644e-3ee2-46cd-96df-c0d3bee79419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a13adbc63c74be6871c638f05c23395'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:13.968444 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ecd3644e-3ee2-46cd-96df-c0d3bee79419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a13adbc63c74be6871c638f05c23395'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:13.983110 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1092/2175] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:13 2022] DELETE /identity/v3/users/8a13adbc63c74be6871c638f05c23395 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:14.420980 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadfaa4b-505e-42bf-af29-eabfa88791fd admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:14.425388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-dadfaa4b-505e-42bf-af29-eabfa88791fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:14.444342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadfaa4b-505e-42bf-af29-eabfa88791fd admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:14.455432 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:14.569325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0c136-87cc-401d-86b7-a9e08121a7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:14.571042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0c136-87cc-401d-86b7-a9e08121a7a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:14.571587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0c136-87cc-401d-86b7-a9e08121a7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:14.572102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0c136-87cc-401d-86b7-a9e08121a7a4 admin admin] PATH_INFO: `/v3/users/8ccfd694989249a78451b1546ac77105` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:14.575134 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0c136-87cc-401d-86b7-a9e08121a7a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ccfd694989249a78451b1546ac77105)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:14.582217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0c136-87cc-401d-86b7-a9e08121a7a4 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:14.725405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f8e0c136-87cc-401d-86b7-a9e08121a7a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ccfd694989249a78451b1546ac77105'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:14.755700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f8e0c136-87cc-401d-86b7-a9e08121a7a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ccfd694989249a78451b1546ac77105'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:14.777488 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3644e-3ee2-46cd-96df-c0d3bee79419 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:14.781030 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ecd3644e-3ee2-46cd-96df-c0d3bee79419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:14.783289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f8e0c136-87cc-401d-86b7-a9e08121a7a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ccfd694989249a78451b1546ac77105'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:14.784335 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f8e0c136-87cc-401d-86b7-a9e08121a7a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ccfd694989249a78451b1546ac77105'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:14.785043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f8e0c136-87cc-401d-86b7-a9e08121a7a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ccfd694989249a78451b1546ac77105'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:14.807805 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1084/2176] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:14 2022] DELETE /identity/v3/users/8ccfd694989249a78451b1546ac77105 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:14.808965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3644e-3ee2-46cd-96df-c0d3bee79419 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:14.814795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cda84-314f-41ea-b0be-5dbd3a0ae586 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:14.815132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cda84-314f-41ea-b0be-5dbd3a0ae586 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:14.815472 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cda84-314f-41ea-b0be-5dbd3a0ae586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:14.883117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b63cda84-314f-41ea-b0be-5dbd3a0ae586 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:14.884220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b63cda84-314f-41ea-b0be-5dbd3a0ae586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:15.040104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e44da5-9a30-496a-845f-41fb81730906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:15.041092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e44da5-9a30-496a-845f-41fb81730906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:15.041509 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e44da5-9a30-496a-845f-41fb81730906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:15.041950 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e44da5-9a30-496a-845f-41fb81730906 admin admin] PATH_INFO: `/v3/projects/ccd3bc56f12b41d09806e8bc10b04ca5` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:15.060597 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e44da5-9a30-496a-845f-41fb81730906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccd3bc56f12b41d09806e8bc10b04ca5)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:15.070287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e44da5-9a30-496a-845f-41fb81730906 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:15.130125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1085/2177] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:15.166836 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cda84-314f-41ea-b0be-5dbd3a0ae586 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:15.169496 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b63cda84-314f-41ea-b0be-5dbd3a0ae586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:15.289656 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:15.332047 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-24e44da5-9a30-496a-845f-41fb81730906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccd3bc56f12b41d09806e8bc10b04ca5', 'request_id': 'req-24e44da5-9a30-496a-845f-41fb81730906'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:15.363824 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-24e44da5-9a30-496a-845f-41fb81730906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccd3bc56f12b41d09806e8bc10b04ca5', 'request_id': 'req-24e44da5-9a30-496a-845f-41fb81730906'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:15.388262 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:15.399596 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cda84-314f-41ea-b0be-5dbd3a0ae586 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:15.414205 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-24e44da5-9a30-496a-845f-41fb81730906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccd3bc56f12b41d09806e8bc10b04ca5', 'request_id': 'req-24e44da5-9a30-496a-845f-41fb81730906'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:15.457144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-24e44da5-9a30-496a-845f-41fb81730906 admin admin] The token cache is being invalidate because project ccd3bc56f12b41d09806e8bc10b04ca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:15.457619 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-24e44da5-9a30-496a-845f-41fb81730906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:15.461386 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-24e44da5-9a30-496a-845f-41fb81730906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:15.472749 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1093/2178] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:14 2022] DELETE /identity/v3/projects/ccd3bc56f12b41d09806e8bc10b04ca5 => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:15.480142 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80af70-0aeb-4012-afad-9c22b55f8d10 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:15.481334 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80af70-0aeb-4012-afad-9c22b55f8d10 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:15.482203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80af70-0aeb-4012-afad-9c22b55f8d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:15.538566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7b80af70-0aeb-4012-afad-9c22b55f8d10 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:15.542572 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7b80af70-0aeb-4012-afad-9c22b55f8d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:15.544874 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1d506-8654-47ac-ab18-486fdd786f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:15.545698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1d506-8654-47ac-ab18-486fdd786f40 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:15.545863 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1d506-8654-47ac-ab18-486fdd786f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:15.546240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1d506-8654-47ac-ab18-486fdd786f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:15.552832 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-81d1d506-8654-47ac-ab18-486fdd786f40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:15.726507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1094/2179] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:16:15.743704 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0eafb8-1322-47b5-b2b8-1ee632ce3368 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:15.744561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0eafb8-1322-47b5-b2b8-1ee632ce3368 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:15.745390 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0eafb8-1322-47b5-b2b8-1ee632ce3368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:15.772265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d1d506-8654-47ac-ab18-486fdd786f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:15.775321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d1d506-8654-47ac-ab18-486fdd786f40 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:15.801128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8f0eafb8-1322-47b5-b2b8-1ee632ce3368 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:15.803937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8f0eafb8-1322-47b5-b2b8-1ee632ce3368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:15.838298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1095/2180] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:15.843711 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0eafb8-1322-47b5-b2b8-1ee632ce3368 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:15.854466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1086/2181] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:16:15.890626 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0eafb8-1322-47b5-b2b8-1ee632ce3368 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:15.914740 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d77697-66fd-41e8-a7e4-0e4fa6f9e829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:15.915791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d77697-66fd-41e8-a7e4-0e4fa6f9e829 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:15.916260 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d77697-66fd-41e8-a7e4-0e4fa6f9e829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:15.916929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d77697-66fd-41e8-a7e4-0e4fa6f9e829 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:15.918377 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d77697-66fd-41e8-a7e4-0e4fa6f9e829 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:16:15.919932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d77697-66fd-41e8-a7e4-0e4fa6f9e829 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:15.927104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d77697-66fd-41e8-a7e4-0e4fa6f9e829 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:15.973005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1096/2182] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:16:15.980627 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1d506-8654-47ac-ab18-486fdd786f40 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:16.006886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1d506-8654-47ac-ab18-486fdd786f40 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:16.026778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3c384-d642-4904-8e07-f79724be778a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:16.030156 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3c384-d642-4904-8e07-f79724be778a admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:16.030705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3c384-d642-4904-8e07-f79724be778a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:16.031262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3c384-d642-4904-8e07-f79724be778a admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:16.043579 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3c384-d642-4904-8e07-f79724be778a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:16.050652 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3c384-d642-4904-8e07-f79724be778a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:16.162496 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1087/2183] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:15 2022] POST /identity/v3/projects => generated 371 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:16.166573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d77697-66fd-41e8-a7e4-0e4fa6f9e829 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:16.232958 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d77697-66fd-41e8-a7e4-0e4fa6f9e829 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:16.408046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd3b9b-0640-4317-95a4-0ea15a0632d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:16.411328 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd3b9b-0640-4317-95a4-0ea15a0632d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:16.411782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd3b9b-0640-4317-95a4-0ea15a0632d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:16.412267 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd3b9b-0640-4317-95a4-0ea15a0632d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:16.414215 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd3b9b-0640-4317-95a4-0ea15a0632d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:16.420522 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd3b9b-0640-4317-95a4-0ea15a0632d6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:16.460156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1097/2184] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:16 2022] POST /identity/v3/users => generated 286 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:16.464288 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3c384-d642-4904-8e07-f79724be778a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:16.477786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3c384-d642-4904-8e07-f79724be778a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:16.498006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de44a3-59ff-484a-835a-2d864f085d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:16.498638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de44a3-59ff-484a-835a-2d864f085d29 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:16.499134 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de44a3-59ff-484a-835a-2d864f085d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:16.499616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de44a3-59ff-484a-835a-2d864f085d29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:16.500500 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de44a3-59ff-484a-835a-2d864f085d29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:16.503337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de44a3-59ff-484a-835a-2d864f085d29 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:16.519329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1088/2185] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:16.524091 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd3b9b-0640-4317-95a4-0ea15a0632d6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:16.539084 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd3b9b-0640-4317-95a4-0ea15a0632d6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:16.545213 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f78ed-d14a-4c1b-9d5c-a8893f418763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:16.545981 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f78ed-d14a-4c1b-9d5c-a8893f418763 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:16.546323 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f78ed-d14a-4c1b-9d5c-a8893f418763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:16.546601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f78ed-d14a-4c1b-9d5c-a8893f418763 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:16.547294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f78ed-d14a-4c1b-9d5c-a8893f418763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:16.549368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f78ed-d14a-4c1b-9d5c-a8893f418763 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:16.562818 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1098/2186] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:16.569705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de44a3-59ff-484a-835a-2d864f085d29 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:16.587542 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de44a3-59ff-484a-835a-2d864f085d29 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:16.594160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3642cf-9c34-466e-af15-b3fedf4f287b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:16.595417 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3642cf-9c34-466e-af15-b3fedf4f287b admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:16.597007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3642cf-9c34-466e-af15-b3fedf4f287b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:16.597007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3642cf-9c34-466e-af15-b3fedf4f287b admin admin] PATH_INFO: `/v3/projects/8eb99d8d2cf64284b325f959f847e4cd/users/37ebd24ddf7b4eebbe31972b82aba518/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:16.623546 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3642cf-9c34-466e-af15-b3fedf4f287b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb99d8d2cf64284b325f959f847e4cd, user_id=37ebd24ddf7b4eebbe31972b82aba518, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:16.628705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3642cf-9c34-466e-af15-b3fedf4f287b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:16.650349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1089/2187] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:16 2022] PUT /identity/v3/projects/8eb99d8d2cf64284b325f959f847e4cd/users/37ebd24ddf7b4eebbe31972b82aba518/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:16.663033 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a21f6-1d70-4080-9565-f0e0fc6bfa31 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:16.663492 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a21f6-1d70-4080-9565-f0e0fc6bfa31 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:16.663868 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a21f6-1d70-4080-9565-f0e0fc6bfa31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:16.711917 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3642cf-9c34-466e-af15-b3fedf4f287b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:16.714940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fa3642cf-9c34-466e-af15-b3fedf4f287b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:16.716009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c53a21f6-1d70-4080-9565-f0e0fc6bfa31 None None] MFA Rules not processed for user `37ebd24ddf7b4eebbe31972b82aba518`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:16.717570 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c53a21f6-1d70-4080-9565-f0e0fc6bfa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:16.802544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:16.854557 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:16.882707 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1099/2188] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:16.897665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a21f6-1d70-4080-9565-f0e0fc6bfa31 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:16.900724 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c53a21f6-1d70-4080-9565-f0e0fc6bfa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:16.910294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3642cf-9c34-466e-af15-b3fedf4f287b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:16.945270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8aa9a-2919-440f-af23-1911ccd94593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:16.950825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8aa9a-2919-440f-af23-1911ccd94593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:16.951133 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8aa9a-2919-440f-af23-1911ccd94593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:16.951543 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8aa9a-2919-440f-af23-1911ccd94593 admin admin] PATH_INFO: `/v3/projects/44b046e5296a4438804420c65973da53` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:16.959876 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8aa9a-2919-440f-af23-1911ccd94593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44b046e5296a4438804420c65973da53)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:16.984296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8aa9a-2919-440f-af23-1911ccd94593 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:17.189860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a21f6-1d70-4080-9565-f0e0fc6bfa31 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:17.190913 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2aa8aa9a-2919-440f-af23-1911ccd94593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44b046e5296a4438804420c65973da53', 'request_id': 'req-2aa8aa9a-2919-440f-af23-1911ccd94593'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:17.205360 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2aa8aa9a-2919-440f-af23-1911ccd94593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44b046e5296a4438804420c65973da53', 'request_id': 'req-2aa8aa9a-2919-440f-af23-1911ccd94593'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:17.217276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2aa8aa9a-2919-440f-af23-1911ccd94593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44b046e5296a4438804420c65973da53', 'request_id': 'req-2aa8aa9a-2919-440f-af23-1911ccd94593'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:17.237267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2aa8aa9a-2919-440f-af23-1911ccd94593 admin admin] The token cache is being invalidate because project 44b046e5296a4438804420c65973da53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:17.238005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2aa8aa9a-2919-440f-af23-1911ccd94593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:17.239153 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2aa8aa9a-2919-440f-af23-1911ccd94593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:17.241429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1090/2189] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:16 2022] DELETE /identity/v3/projects/44b046e5296a4438804420c65973da53 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:16:17.246230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8aa9a-2919-440f-af23-1911ccd94593 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:17.249706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2aa8aa9a-2919-440f-af23-1911ccd94593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:17.301877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8cff3-0d73-4438-b6e9-2128cef77448 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:17.302562 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8cff3-0d73-4438-b6e9-2128cef77448 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:17.302917 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8cff3-0d73-4438-b6e9-2128cef77448 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:17.303188 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8cff3-0d73-4438-b6e9-2128cef77448 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:17.306293 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5ed8cff3-0d73-4438-b6e9-2128cef77448 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:17.391491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8aa9a-2919-440f-af23-1911ccd94593 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:17.403804 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb30687-c5aa-44c4-b448-16c6a6c0ae7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:17.404472 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb30687-c5aa-44c4-b448-16c6a6c0ae7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:17.404757 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb30687-c5aa-44c4-b448-16c6a6c0ae7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:17.405036 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb30687-c5aa-44c4-b448-16c6a6c0ae7f admin admin] PATH_INFO: `/v3/users/8b28d478ad514d2292b141649dfe7227` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:17.406507 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb30687-c5aa-44c4-b448-16c6a6c0ae7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b28d478ad514d2292b141649dfe7227)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:17.410878 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb30687-c5aa-44c4-b448-16c6a6c0ae7f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:17.424264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8cff3-0d73-4438-b6e9-2128cef77448 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:17.425930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8cff3-0d73-4438-b6e9-2128cef77448 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:17.452625 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1100/2190] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:17.496540 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8cff3-0d73-4438-b6e9-2128cef77448 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:17.499190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5ed8cff3-0d73-4438-b6e9-2128cef77448 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:17.529384 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8cff3-0d73-4438-b6e9-2128cef77448 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:17.538831 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2cb30687-c5aa-44c4-b448-16c6a6c0ae7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b28d478ad514d2292b141649dfe7227'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:17.540721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1cad5-341e-40a1-8c45-faf3b47f7f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:17.541496 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1cad5-341e-40a1-8c45-faf3b47f7f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:17.542063 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1cad5-341e-40a1-8c45-faf3b47f7f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:17.542700 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1cad5-341e-40a1-8c45-faf3b47f7f23 admin admin] PATH_INFO: `/v3/users/37ebd24ddf7b4eebbe31972b82aba518` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:17.546563 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1cad5-341e-40a1-8c45-faf3b47f7f23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37ebd24ddf7b4eebbe31972b82aba518)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:17.551249 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1cad5-341e-40a1-8c45-faf3b47f7f23 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:17.586718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2cb30687-c5aa-44c4-b448-16c6a6c0ae7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b28d478ad514d2292b141649dfe7227'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:17.634774 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:17.639515 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2cb30687-c5aa-44c4-b448-16c6a6c0ae7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b28d478ad514d2292b141649dfe7227'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:17.643195 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2cb30687-c5aa-44c4-b448-16c6a6c0ae7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b28d478ad514d2292b141649dfe7227'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:17.645530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2cb30687-c5aa-44c4-b448-16c6a6c0ae7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b28d478ad514d2292b141649dfe7227'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:17.669061 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:17.677423 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1091/2191] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:17 2022] DELETE /identity/v3/users/8b28d478ad514d2292b141649dfe7227 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:17.689998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c0e1cad5-341e-40a1-8c45-faf3b47f7f23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37ebd24ddf7b4eebbe31972b82aba518'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:17.701773 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c0e1cad5-341e-40a1-8c45-faf3b47f7f23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37ebd24ddf7b4eebbe31972b82aba518'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:17.724073 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c0e1cad5-341e-40a1-8c45-faf3b47f7f23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37ebd24ddf7b4eebbe31972b82aba518'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:17.725221 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c0e1cad5-341e-40a1-8c45-faf3b47f7f23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37ebd24ddf7b4eebbe31972b82aba518'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:17.739316 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c0e1cad5-341e-40a1-8c45-faf3b47f7f23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37ebd24ddf7b4eebbe31972b82aba518'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:17.754847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187760c0-e9ac-4ea7-80d1-d61f1b0cadae None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:17.755153 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187760c0-e9ac-4ea7-80d1-d61f1b0cadae None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:17.755438 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187760c0-e9ac-4ea7-80d1-d61f1b0cadae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:17.760094 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1101/2192] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:17 2022] DELETE /identity/v3/users/37ebd24ddf7b4eebbe31972b82aba518 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:17.806355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-187760c0-e9ac-4ea7-80d1-d61f1b0cadae None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:17.809692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-187760c0-e9ac-4ea7-80d1-d61f1b0cadae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:18.049695 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1092/2193] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:18.079363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1cad5-341e-40a1-8c45-faf3b47f7f23 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:18.082135 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c0e1cad5-341e-40a1-8c45-faf3b47f7f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:18.248002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1cad5-341e-40a1-8c45-faf3b47f7f23 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:18.280110 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37255d70-a631-4b61-a286-366b124d7bd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:18.285499 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37255d70-a631-4b61-a286-366b124d7bd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:18.286216 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37255d70-a631-4b61-a286-366b124d7bd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:18.286529 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37255d70-a631-4b61-a286-366b124d7bd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:18.309869 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37255d70-a631-4b61-a286-366b124d7bd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:18.311444 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37255d70-a631-4b61-a286-366b124d7bd6 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:18.334268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1102/2194] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:18.497705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187760c0-e9ac-4ea7-80d1-d61f1b0cadae None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:18.503483 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-187760c0-e9ac-4ea7-80d1-d61f1b0cadae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:18.566236 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187760c0-e9ac-4ea7-80d1-d61f1b0cadae None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:18.577460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3da77-7914-4738-84f8-6e2d2e431568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:18.580662 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3da77-7914-4738-84f8-6e2d2e431568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:18.581557 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3da77-7914-4738-84f8-6e2d2e431568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:18.582625 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3da77-7914-4738-84f8-6e2d2e431568 admin admin] PATH_INFO: `/v3/projects/f194ccf713ec49e1bf1b98345afb8650` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:18.585907 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3da77-7914-4738-84f8-6e2d2e431568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f194ccf713ec49e1bf1b98345afb8650)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:18.590390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3da77-7914-4738-84f8-6e2d2e431568 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:18.806269 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-44d3da77-7914-4738-84f8-6e2d2e431568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f194ccf713ec49e1bf1b98345afb8650', 'request_id': 'req-44d3da77-7914-4738-84f8-6e2d2e431568'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:18.819543 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37255d70-a631-4b61-a286-366b124d7bd6 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:18.821705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-37255d70-a631-4b61-a286-366b124d7bd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:18.833553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-44d3da77-7914-4738-84f8-6e2d2e431568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f194ccf713ec49e1bf1b98345afb8650', 'request_id': 'req-44d3da77-7914-4738-84f8-6e2d2e431568'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:18.841063 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37255d70-a631-4b61-a286-366b124d7bd6 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:18.843289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-44d3da77-7914-4738-84f8-6e2d2e431568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f194ccf713ec49e1bf1b98345afb8650', 'request_id': 'req-44d3da77-7914-4738-84f8-6e2d2e431568'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:18.883367 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-44d3da77-7914-4738-84f8-6e2d2e431568 admin admin] The token cache is being invalidate because project f194ccf713ec49e1bf1b98345afb8650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:18.883949 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-44d3da77-7914-4738-84f8-6e2d2e431568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:18.885424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-44d3da77-7914-4738-84f8-6e2d2e431568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:18.888391 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1093/2195] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:18 2022] DELETE /identity/v3/projects/f194ccf713ec49e1bf1b98345afb8650 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:18.895440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3e03a-2005-4898-a1ff-2f0f9be902e3 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:18.898909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3e03a-2005-4898-a1ff-2f0f9be902e3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:18.899393 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3e03a-2005-4898-a1ff-2f0f9be902e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:18.945982 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-eeb3e03a-2005-4898-a1ff-2f0f9be902e3 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:18.948514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-eeb3e03a-2005-4898-a1ff-2f0f9be902e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:19.009570 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e65c2f-e7f0-46cb-9351-c6fd19b4f8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:19.010124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e65c2f-e7f0-46cb-9351-c6fd19b4f8fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:19.011336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e65c2f-e7f0-46cb-9351-c6fd19b4f8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:19.011336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e65c2f-e7f0-46cb-9351-c6fd19b4f8fd admin admin] PATH_INFO: `/v3/projects/8eb99d8d2cf64284b325f959f847e4cd` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:19.014242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e65c2f-e7f0-46cb-9351-c6fd19b4f8fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eb99d8d2cf64284b325f959f847e4cd)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:19.017118 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e65c2f-e7f0-46cb-9351-c6fd19b4f8fd admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:19.111007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1094/2196] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:19.120388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca669d-c8af-4eda-8e93-c781abcfd681 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:19.121516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca669d-c8af-4eda-8e93-c781abcfd681 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:19.122581 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca669d-c8af-4eda-8e93-c781abcfd681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:19.129918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-05e65c2f-e7f0-46cb-9351-c6fd19b4f8fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eb99d8d2cf64284b325f959f847e4cd', 'request_id': 'req-05e65c2f-e7f0-46cb-9351-c6fd19b4f8fd'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:19.141011 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-05e65c2f-e7f0-46cb-9351-c6fd19b4f8fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eb99d8d2cf64284b325f959f847e4cd', 'request_id': 'req-05e65c2f-e7f0-46cb-9351-c6fd19b4f8fd'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:19.152036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:19.153915 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-05e65c2f-e7f0-46cb-9351-c6fd19b4f8fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eb99d8d2cf64284b325f959f847e4cd', 'request_id': 'req-05e65c2f-e7f0-46cb-9351-c6fd19b4f8fd'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:19.176788 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d9ca669d-c8af-4eda-8e93-c781abcfd681 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:19.179169 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d9ca669d-c8af-4eda-8e93-c781abcfd681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:19.183101 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-05e65c2f-e7f0-46cb-9351-c6fd19b4f8fd admin admin] The token cache is being invalidate because project 8eb99d8d2cf64284b325f959f847e4cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:19.183388 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-05e65c2f-e7f0-46cb-9351-c6fd19b4f8fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:19.184319 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-05e65c2f-e7f0-46cb-9351-c6fd19b4f8fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:19.188034 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1103/2197] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:18 2022] DELETE /identity/v3/projects/8eb99d8d2cf64284b325f959f847e4cd => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:19.192324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:19.196389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2755e-d2a4-41fb-82e9-8dbcef57c0ee None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:19.196677 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2755e-d2a4-41fb-82e9-8dbcef57c0ee None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:19.196955 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2755e-d2a4-41fb-82e9-8dbcef57c0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:19.221946 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-95c2755e-d2a4-41fb-82e9-8dbcef57c0ee None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:19.223219 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-95c2755e-d2a4-41fb-82e9-8dbcef57c0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:19.280890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1095/2198] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:19.282765 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca669d-c8af-4eda-8e93-c781abcfd681 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:19.308049 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca669d-c8af-4eda-8e93-c781abcfd681 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:19.318702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4043625b-22e9-4b87-a9e3-18a2cfca87cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:19.319919 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4043625b-22e9-4b87-a9e3-18a2cfca87cd admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:19.322569 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4043625b-22e9-4b87-a9e3-18a2cfca87cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:19.323276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4043625b-22e9-4b87-a9e3-18a2cfca87cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:19.324633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4043625b-22e9-4b87-a9e3-18a2cfca87cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:16:19.325682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4043625b-22e9-4b87-a9e3-18a2cfca87cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:19.328643 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1104/2199] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:19.332694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4043625b-22e9-4b87-a9e3-18a2cfca87cd admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:19.338900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250889ee-f073-4b2c-a207-6ac1dc9759fa None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:19.339580 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250889ee-f073-4b2c-a207-6ac1dc9759fa None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:19.340038 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250889ee-f073-4b2c-a207-6ac1dc9759fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:19.353940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1096/2200] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:16:19.361874 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4043625b-22e9-4b87-a9e3-18a2cfca87cd admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:19.379907 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4043625b-22e9-4b87-a9e3-18a2cfca87cd admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:19.386377 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a541b25-ce80-4dc8-9913-2970113de7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:19.387413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a541b25-ce80-4dc8-9913-2970113de7cd admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:19.387696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a541b25-ce80-4dc8-9913-2970113de7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:19.388001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a541b25-ce80-4dc8-9913-2970113de7cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:19.388997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a541b25-ce80-4dc8-9913-2970113de7cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:19.391895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-250889ee-f073-4b2c-a207-6ac1dc9759fa None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:19.393572 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a541b25-ce80-4dc8-9913-2970113de7cd admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:19.394877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-250889ee-f073-4b2c-a207-6ac1dc9759fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:19.413772 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1105/2201] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:19.416329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250889ee-f073-4b2c-a207-6ac1dc9759fa None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:19.443504 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250889ee-f073-4b2c-a207-6ac1dc9759fa None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:19.444550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1097/2202] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:19 2022] POST /identity/v3/projects => generated 383 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:19.450151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a541b25-ce80-4dc8-9913-2970113de7cd admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:19.537421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f99f4-3282-4676-8e85-168f564b8607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:19.537938 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f99f4-3282-4676-8e85-168f564b8607 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:19.538219 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f99f4-3282-4676-8e85-168f564b8607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:19.538496 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f99f4-3282-4676-8e85-168f564b8607 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:19.539201 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f99f4-3282-4676-8e85-168f564b8607 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:16:19.539516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f99f4-3282-4676-8e85-168f564b8607 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:19.541199 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f99f4-3282-4676-8e85-168f564b8607 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:19.566209 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1106/2203] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:16:19.567392 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f99f4-3282-4676-8e85-168f564b8607 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:19.580417 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f99f4-3282-4676-8e85-168f564b8607 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:19.581948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a541b25-ce80-4dc8-9913-2970113de7cd admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:19.586501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfae2de-22c3-453e-b4fa-8bb70708f387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:19.587272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfae2de-22c3-453e-b4fa-8bb70708f387 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:19.587620 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfae2de-22c3-453e-b4fa-8bb70708f387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:19.587895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfae2de-22c3-453e-b4fa-8bb70708f387 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:19.588545 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfae2de-22c3-453e-b4fa-8bb70708f387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:19.590813 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666ece9-6425-478e-8de8-dea11dc4aa49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:19.591320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfae2de-22c3-453e-b4fa-8bb70708f387 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:19.591841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666ece9-6425-478e-8de8-dea11dc4aa49 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:19.592308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666ece9-6425-478e-8de8-dea11dc4aa49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:19.592776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666ece9-6425-478e-8de8-dea11dc4aa49 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:19.593940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666ece9-6425-478e-8de8-dea11dc4aa49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:19.598443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666ece9-6425-478e-8de8-dea11dc4aa49 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:19.632696 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1107/2204] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:19 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:19.633753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1098/2205] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:19 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:16:19.638972 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666ece9-6425-478e-8de8-dea11dc4aa49 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:19.639180 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfae2de-22c3-453e-b4fa-8bb70708f387 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:19.651408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666ece9-6425-478e-8de8-dea11dc4aa49 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:19.711936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:19.732831 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfae2de-22c3-453e-b4fa-8bb70708f387 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:19.748391 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b1814-7636-4647-99bf-5be1fbfa605e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:19.749816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b1814-7636-4647-99bf-5be1fbfa605e admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:19.750184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b1814-7636-4647-99bf-5be1fbfa605e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:19.751576 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b1814-7636-4647-99bf-5be1fbfa605e admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:19.752723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b1814-7636-4647-99bf-5be1fbfa605e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:19.754439 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6471ef4d-89a6-45dd-bf69-b0f7d1afe430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:19.755137 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471ef4d-89a6-45dd-bf69-b0f7d1afe430 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:19.755417 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471ef4d-89a6-45dd-bf69-b0f7d1afe430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:19.755666 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b1814-7636-4647-99bf-5be1fbfa605e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:19.755847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471ef4d-89a6-45dd-bf69-b0f7d1afe430 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:19.756342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6471ef4d-89a6-45dd-bf69-b0f7d1afe430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:19.762561 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6471ef4d-89a6-45dd-bf69-b0f7d1afe430 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:19.781188 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1099/2206] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:19.783203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ec96a-f054-4ac8-b927-37d9c46acaae None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:19.783642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ec96a-f054-4ac8-b927-37d9c46acaae None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:19.783941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ec96a-f054-4ac8-b927-37d9c46acaae None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:19.785539 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1100/2207] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 6 16:16:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:16:19.786743 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ec96a-f054-4ac8-b927-37d9c46acaae None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:19.788845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-808ec96a-f054-4ac8-b927-37d9c46acaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:19.791977 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1108/2208] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:19 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:19.794041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b1814-7636-4647-99bf-5be1fbfa605e admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:19.799619 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:19.808002 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ec96a-f054-4ac8-b927-37d9c46acaae None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:19.814267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8595d-d770-4916-be42-ef43b2077e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:19.814949 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8595d-d770-4916-be42-ef43b2077e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:19.815231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8595d-d770-4916-be42-ef43b2077e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:19.815511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8595d-d770-4916-be42-ef43b2077e67 admin admin] PATH_INFO: `/v3/users/668452994ad942069d30f4bddbba15b4` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:19.816330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b1814-7636-4647-99bf-5be1fbfa605e admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:19.816720 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff8595d-d770-4916-be42-ef43b2077e67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=668452994ad942069d30f4bddbba15b4)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:19.818766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff8595d-d770-4916-be42-ef43b2077e67 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:19.823778 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4236a-94cd-4ff3-88c2-80164cef8bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:19.824859 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4236a-94cd-4ff3-88c2-80164cef8bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:19.825162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4236a-94cd-4ff3-88c2-80164cef8bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:19.825555 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4236a-94cd-4ff3-88c2-80164cef8bf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:19.826692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4236a-94cd-4ff3-88c2-80164cef8bf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:19.829871 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4236a-94cd-4ff3-88c2-80164cef8bf2 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:19.849972 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1109/2209] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:19.852680 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fe114-a67a-4926-bf63-f2c5f3cbbdda None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:19.853307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fe114-a67a-4926-bf63-f2c5f3cbbdda None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:19.853715 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fe114-a67a-4926-bf63-f2c5f3cbbdda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:19.879126 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1ff8595d-d770-4916-be42-ef43b2077e67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '668452994ad942069d30f4bddbba15b4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:19.884977 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f68fe114-a67a-4926-bf63-f2c5f3cbbdda None None] MFA Rules not processed for user `034d75ab747e4c0cab040e47c6a6236e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:19.886286 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f68fe114-a67a-4926-bf63-f2c5f3cbbdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:19.887937 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1ff8595d-d770-4916-be42-ef43b2077e67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '668452994ad942069d30f4bddbba15b4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:19.909863 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1ff8595d-d770-4916-be42-ef43b2077e67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '668452994ad942069d30f4bddbba15b4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:19.911731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1ff8595d-d770-4916-be42-ef43b2077e67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '668452994ad942069d30f4bddbba15b4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:19.912663 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1ff8595d-d770-4916-be42-ef43b2077e67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '668452994ad942069d30f4bddbba15b4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:19.932530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1101/2210] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:19 2022] DELETE /identity/v3/users/668452994ad942069d30f4bddbba15b4 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:19.933409 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8595d-d770-4916-be42-ef43b2077e67 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:19.935060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1ff8595d-d770-4916-be42-ef43b2077e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:20.031594 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1110/2211] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 6 16:16:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:20.035526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fe114-a67a-4926-bf63-f2c5f3cbbdda None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:20.040202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f68fe114-a67a-4926-bf63-f2c5f3cbbdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:20.058694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8595d-d770-4916-be42-ef43b2077e67 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:20.066190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fe114-a67a-4926-bf63-f2c5f3cbbdda None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:20.068455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638a408-ceb3-4544-ab53-d4e9815e8a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:20.069522 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638a408-ceb3-4544-ab53-d4e9815e8a48 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:20.070779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638a408-ceb3-4544-ab53-d4e9815e8a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:20.070779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638a408-ceb3-4544-ab53-d4e9815e8a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:20.071905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638a408-ceb3-4544-ab53-d4e9815e8a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:20.074652 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638a408-ceb3-4544-ab53-d4e9815e8a48 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:20.075619 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c15462-795c-414d-87aa-f77f0e694c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:20.076773 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c15462-795c-414d-87aa-f77f0e694c7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:20.077251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c15462-795c-414d-87aa-f77f0e694c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:20.077730 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c15462-795c-414d-87aa-f77f0e694c7e admin admin] PATH_INFO: `/v3/projects/8550a73cb68548218b601f20fd8d1668/users/0d4464700613439b9dc2b3864e67a695/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:20.095731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1102/2212] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:20.098118 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e2d0d-bb38-43fe-af99-06833ca3c899 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:20.098630 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e2d0d-bb38-43fe-af99-06833ca3c899 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:20.099204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e2d0d-bb38-43fe-af99-06833ca3c899 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:20.101750 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1103/2213] 10.222.0.26 () {54 vars in 961 bytes} [Tue Sep 6 16:16:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:16:20.103262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e2d0d-bb38-43fe-af99-06833ca3c899 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:20.106414 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c15462-795c-414d-87aa-f77f0e694c7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8550a73cb68548218b601f20fd8d1668, user_id=0d4464700613439b9dc2b3864e67a695, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:20.111791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c15462-795c-414d-87aa-f77f0e694c7e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:20.128754 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e2d0d-bb38-43fe-af99-06833ca3c899 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:20.136279 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1111/2214] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:20 2022] PUT /identity/v3/projects/8550a73cb68548218b601f20fd8d1668/users/0d4464700613439b9dc2b3864e67a695/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:20.138485 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ae874-5720-43c7-b6ce-f1ea2f18078c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:20.139014 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ae874-5720-43c7-b6ce-f1ea2f18078c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:20.139978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ae874-5720-43c7-b6ce-f1ea2f18078c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:20.171210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-9a3ae874-5720-43c7-b6ce-f1ea2f18078c None None] MFA Rules not processed for user `034d75ab747e4c0cab040e47c6a6236e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:20.174652 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9a3ae874-5720-43c7-b6ce-f1ea2f18078c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:20.299735 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21926e-f388-4830-8ea3-644c51ee4ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:20.301428 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21926e-f388-4830-8ea3-644c51ee4ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:20.301931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21926e-f388-4830-8ea3-644c51ee4ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:20.302424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21926e-f388-4830-8ea3-644c51ee4ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:20.304317 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da21926e-f388-4830-8ea3-644c51ee4ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:20.310802 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da21926e-f388-4830-8ea3-644c51ee4ce8 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:20.334111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1104/2215] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:20.335923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7ff6e-7d1f-4829-9b5b-4c1d29736597 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:20.336376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7ff6e-7d1f-4829-9b5b-4c1d29736597 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:20.336832 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7ff6e-7d1f-4829-9b5b-4c1d29736597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:20.338536 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1112/2216] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Sep 6 16:16:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:20.339468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ae874-5720-43c7-b6ce-f1ea2f18078c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:20.341058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9a3ae874-5720-43c7-b6ce-f1ea2f18078c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:20.355732 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:20.358912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ae874-5720-43c7-b6ce-f1ea2f18078c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:20.365129 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b7cf8-e623-44ba-993f-e8eddd7f9c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:20.365781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b7cf8-e623-44ba-993f-e8eddd7f9c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:20.366064 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b7cf8-e623-44ba-993f-e8eddd7f9c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:20.366330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b7cf8-e623-44ba-993f-e8eddd7f9c86 admin admin] PATH_INFO: `/v3/projects/c5974f0b22a7493da25b0b0aaa21c66b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:20.367556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b7cf8-e623-44ba-993f-e8eddd7f9c86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5974f0b22a7493da25b0b0aaa21c66b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:20.369492 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b7cf8-e623-44ba-993f-e8eddd7f9c86 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:20.380848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b7b7ff6e-7d1f-4829-9b5b-4c1d29736597 None None] MFA Rules not processed for user `0d4464700613439b9dc2b3864e67a695`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:20.382796 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b7b7ff6e-7d1f-4829-9b5b-4c1d29736597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:20.433616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:20.455391 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a84b7cf8-e623-44ba-993f-e8eddd7f9c86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5974f0b22a7493da25b0b0aaa21c66b', 'request_id': 'req-a84b7cf8-e623-44ba-993f-e8eddd7f9c86'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:20.464733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a84b7cf8-e623-44ba-993f-e8eddd7f9c86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5974f0b22a7493da25b0b0aaa21c66b', 'request_id': 'req-a84b7cf8-e623-44ba-993f-e8eddd7f9c86'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:20.473624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a84b7cf8-e623-44ba-993f-e8eddd7f9c86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5974f0b22a7493da25b0b0aaa21c66b', 'request_id': 'req-a84b7cf8-e623-44ba-993f-e8eddd7f9c86'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:20.474699 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1105/2217] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:20.475844 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7ff6e-7d1f-4829-9b5b-4c1d29736597 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:20.491232 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a84b7cf8-e623-44ba-993f-e8eddd7f9c86 admin admin] The token cache is being invalidate because project c5974f0b22a7493da25b0b0aaa21c66b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:20.491529 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a84b7cf8-e623-44ba-993f-e8eddd7f9c86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:20.491671 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7ff6e-7d1f-4829-9b5b-4c1d29736597 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:20.492247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a84b7cf8-e623-44ba-993f-e8eddd7f9c86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:20.493799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1113/2218] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:20 2022] DELETE /identity/v3/projects/c5974f0b22a7493da25b0b0aaa21c66b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:20.496125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b7cf8-e623-44ba-993f-e8eddd7f9c86 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:20.498369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a84b7cf8-e623-44ba-993f-e8eddd7f9c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:20.561119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e68c13-9896-4c12-b3e2-d11e56e40617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:20.561981 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e68c13-9896-4c12-b3e2-d11e56e40617 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:20.562556 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e68c13-9896-4c12-b3e2-d11e56e40617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:20.562965 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e68c13-9896-4c12-b3e2-d11e56e40617 admin admin] PATH_INFO: `/v3/projects/d06bd5bc00e54c3b97356760a3f5fa01/users/08e5d528da0f4e89a1882d64b43ecd2b/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:20.581849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e68c13-9896-4c12-b3e2-d11e56e40617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d06bd5bc00e54c3b97356760a3f5fa01, user_id=08e5d528da0f4e89a1882d64b43ecd2b, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:20.584837 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e68c13-9896-4c12-b3e2-d11e56e40617 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:20.612669 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1106/2219] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:20 2022] PUT /identity/v3/projects/d06bd5bc00e54c3b97356760a3f5fa01/users/08e5d528da0f4e89a1882d64b43ecd2b/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:20.614042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e68c13-9896-4c12-b3e2-d11e56e40617 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:20.624677 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-41e68c13-9896-4c12-b3e2-d11e56e40617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:20.646082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b7cf8-e623-44ba-993f-e8eddd7f9c86 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:20.732931 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928d78c-714a-46e3-aa47-56a058505fe1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034d75ab747e4c0cab040e47c6a6236e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:20.733610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928d78c-714a-46e3-aa47-56a058505fe1 service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:20.733884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928d78c-714a-46e3-aa47-56a058505fe1 service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:20.734160 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928d78c-714a-46e3-aa47-56a058505fe1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:20.736577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e928d78c-714a-46e3-aa47-56a058505fe1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:20.745087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e68c13-9896-4c12-b3e2-d11e56e40617 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:20.748709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928d78c-714a-46e3-aa47-56a058505fe1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:20.750336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928d78c-714a-46e3-aa47-56a058505fe1 service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:20.750469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bebd95-2b72-4daa-a3e4-c8f2d5c3d6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:20.751149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bebd95-2b72-4daa-a3e4-c8f2d5c3d6e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:20.751624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bebd95-2b72-4daa-a3e4-c8f2d5c3d6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:20.752102 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bebd95-2b72-4daa-a3e4-c8f2d5c3d6e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:20.752755 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bebd95-2b72-4daa-a3e4-c8f2d5c3d6e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:20.754715 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bebd95-2b72-4daa-a3e4-c8f2d5c3d6e1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:20.761746 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1114/2220] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:20.763001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447d7e2-e75b-4028-a1c5-b693cf02782e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:20.763329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447d7e2-e75b-4028-a1c5-b693cf02782e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:20.763675 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447d7e2-e75b-4028-a1c5-b693cf02782e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:20.789396 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1107/2221] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:20 2022] POST /identity/v3/projects => generated 383 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:20.790814 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa3ed0-d1dc-49e7-802a-93650eb0aa6e None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:20.791097 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa3ed0-d1dc-49e7-802a-93650eb0aa6e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:20.791365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa3ed0-d1dc-49e7-802a-93650eb0aa6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:20.803256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:20.809651 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b447d7e2-e75b-4028-a1c5-b693cf02782e None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:20.810929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b447d7e2-e75b-4028-a1c5-b693cf02782e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:20.830060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-38fa3ed0-d1dc-49e7-802a-93650eb0aa6e None None] MFA Rules not processed for user `08e5d528da0f4e89a1882d64b43ecd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:20.831342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-38fa3ed0-d1dc-49e7-802a-93650eb0aa6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:20.871187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:20.889037 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1115/2222] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:20.890129 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447d7e2-e75b-4028-a1c5-b693cf02782e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:20.903355 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447d7e2-e75b-4028-a1c5-b693cf02782e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:20.909302 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09db11-a973-40bf-be9b-a06b658b88e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:20.910191 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09db11-a973-40bf-be9b-a06b658b88e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:20.910648 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09db11-a973-40bf-be9b-a06b658b88e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:20.911139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09db11-a973-40bf-be9b-a06b658b88e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:20.912443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09db11-a973-40bf-be9b-a06b658b88e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:20.916120 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09db11-a973-40bf-be9b-a06b658b88e1 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:20.927058 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1108/2223] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:20.929110 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5d6ae-22f7-4bd9-ace1-373d4e32b41a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:20.929416 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5d6ae-22f7-4bd9-ace1-373d4e32b41a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:20.929691 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5d6ae-22f7-4bd9-ace1-373d4e32b41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:20.953403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1116/2224] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:20 2022] POST /identity/v3/users => generated 298 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:20.955647 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09db11-a973-40bf-be9b-a06b658b88e1 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:20.957074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-c0f5d6ae-22f7-4bd9-ace1-373d4e32b41a None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:20.958299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c0f5d6ae-22f7-4bd9-ace1-373d4e32b41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:20.959678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3d09db11-a973-40bf-be9b-a06b658b88e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:20.973302 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09db11-a973-40bf-be9b-a06b658b88e1 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:20.978056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1e64d-e801-4357-91eb-3b87bce3a3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:20.978690 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1e64d-e801-4357-91eb-3b87bce3a3be admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:20.978981 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1e64d-e801-4357-91eb-3b87bce3a3be admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:20.979255 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1e64d-e801-4357-91eb-3b87bce3a3be admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:20.979891 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1e64d-e801-4357-91eb-3b87bce3a3be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:20.981179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1109/2225] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:20.981868 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1e64d-e801-4357-91eb-3b87bce3a3be admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:20.982145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5d6ae-22f7-4bd9-ace1-373d4e32b41a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:20.992786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5d6ae-22f7-4bd9-ace1-373d4e32b41a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:20.997533 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc4b2a-320e-4bae-b36e-8ad7b1b9ecf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:20.998139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc4b2a-320e-4bae-b36e-8ad7b1b9ecf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:20.998418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc4b2a-320e-4bae-b36e-8ad7b1b9ecf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:20.998727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc4b2a-320e-4bae-b36e-8ad7b1b9ecf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:20.999375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc4b2a-320e-4bae-b36e-8ad7b1b9ecf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:21.001134 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc4b2a-320e-4bae-b36e-8ad7b1b9ecf3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:21.012249 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1110/2226] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:21.013110 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc4b2a-320e-4bae-b36e-8ad7b1b9ecf3 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:21.018005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1117/2227] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:20 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:21.019874 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1e64d-e801-4357-91eb-3b87bce3a3be admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:21.024512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc4b2a-320e-4bae-b36e-8ad7b1b9ecf3 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:21.099530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03cabe-443a-4c2b-a505-730e616c0bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:21.100030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03cabe-443a-4c2b-a505-730e616c0bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:21.100299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03cabe-443a-4c2b-a505-730e616c0bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:21.100594 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03cabe-443a-4c2b-a505-730e616c0bb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:21.101238 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03cabe-443a-4c2b-a505-730e616c0bb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:16:21.101546 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03cabe-443a-4c2b-a505-730e616c0bb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:21.103242 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03cabe-443a-4c2b-a505-730e616c0bb2 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:21.117019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1111/2228] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:16:21.117861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03cabe-443a-4c2b-a505-730e616c0bb2 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:21.118782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1e64d-e801-4357-91eb-3b87bce3a3be admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:21.124375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5b624-dd49-4bc9-b530-63179cb41842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:21.125126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5b624-dd49-4bc9-b530-63179cb41842 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:21.125458 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5b624-dd49-4bc9-b530-63179cb41842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:21.126412 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5b624-dd49-4bc9-b530-63179cb41842 admin admin] PATH_INFO: `/v3/projects/e278523cbcdd4d19af6f03bc633c9f78/users/fc525cd22ba84eca8d54b4a6ef4eb3cc/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:21.130956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03cabe-443a-4c2b-a505-730e616c0bb2 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:21.136128 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb76005-dff7-47f1-8279-0933224e79b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:21.136750 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb76005-dff7-47f1-8279-0933224e79b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:21.137032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb76005-dff7-47f1-8279-0933224e79b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:21.137300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb76005-dff7-47f1-8279-0933224e79b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:21.138073 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb76005-dff7-47f1-8279-0933224e79b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:21.140099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb76005-dff7-47f1-8279-0933224e79b6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:21.143801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5b624-dd49-4bc9-b530-63179cb41842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e278523cbcdd4d19af6f03bc633c9f78, user_id=fc525cd22ba84eca8d54b4a6ef4eb3cc, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:21.146508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5b624-dd49-4bc9-b530-63179cb41842 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:21.165051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1118/2229] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:21 2022] PUT /identity/v3/projects/e278523cbcdd4d19af6f03bc633c9f78/users/fc525cd22ba84eca8d54b4a6ef4eb3cc/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:21.165886 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5b624-dd49-4bc9-b530-63179cb41842 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:21.166490 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1112/2230] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:21 2022] POST /identity/v3/users => generated 288 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:21.170817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb76005-dff7-47f1-8279-0933224e79b6 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:21.182181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb76005-dff7-47f1-8279-0933224e79b6 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:21.248890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:21.261363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836417bc-05d2-4bdc-a1a8-9fc4cab34a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:21.262558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836417bc-05d2-4bdc-a1a8-9fc4cab34a6c admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:21.263127 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836417bc-05d2-4bdc-a1a8-9fc4cab34a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:21.263519 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5b624-dd49-4bc9-b530-63179cb41842 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:21.263633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836417bc-05d2-4bdc-a1a8-9fc4cab34a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:21.264870 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836417bc-05d2-4bdc-a1a8-9fc4cab34a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:21.268846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e388c0-6eca-4363-a59e-55acd1f62c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:21.269041 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836417bc-05d2-4bdc-a1a8-9fc4cab34a6c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:21.269481 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e388c0-6eca-4363-a59e-55acd1f62c18 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:21.269757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e388c0-6eca-4363-a59e-55acd1f62c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:21.270028 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e388c0-6eca-4363-a59e-55acd1f62c18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:21.270686 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e388c0-6eca-4363-a59e-55acd1f62c18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:21.272640 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e388c0-6eca-4363-a59e-55acd1f62c18 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:21.281271 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1113/2231] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:16:21.286823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836417bc-05d2-4bdc-a1a8-9fc4cab34a6c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:21.305755 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836417bc-05d2-4bdc-a1a8-9fc4cab34a6c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:21.305968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1119/2232] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:21 2022] POST /identity/v3/projects => generated 377 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:21.306917 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e388c0-6eca-4363-a59e-55acd1f62c18 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:21.341572 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:21.379545 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880eea8d-753f-4875-854a-558a926d4c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:21.380236 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880eea8d-753f-4875-854a-558a926d4c08 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:21.380589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880eea8d-753f-4875-854a-558a926d4c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:21.380872 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880eea8d-753f-4875-854a-558a926d4c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:21.381526 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880eea8d-753f-4875-854a-558a926d4c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:21.383187 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880eea8d-753f-4875-854a-558a926d4c08 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:21.394150 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1114/2233] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:21.395062 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880eea8d-753f-4875-854a-558a926d4c08 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:21.395170 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e388c0-6eca-4363-a59e-55acd1f62c18 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:21.403126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef9c89-8390-48ae-b1ff-ca8ee943ee7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:21.404315 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef9c89-8390-48ae-b1ff-ca8ee943ee7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:21.404801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef9c89-8390-48ae-b1ff-ca8ee943ee7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:21.405287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef9c89-8390-48ae-b1ff-ca8ee943ee7f admin admin] PATH_INFO: `/v3/projects/e278523cbcdd4d19af6f03bc633c9f78/users/fc525cd22ba84eca8d54b4a6ef4eb3cc/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:21.405928 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880eea8d-753f-4875-854a-558a926d4c08 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:21.409567 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef9c89-8390-48ae-b1ff-ca8ee943ee7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e278523cbcdd4d19af6f03bc633c9f78, user_id=fc525cd22ba84eca8d54b4a6ef4eb3cc, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:21.411429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f2dc4-348a-4168-94de-2423d4d57146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:21.412043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f2dc4-348a-4168-94de-2423d4d57146 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:21.412326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f2dc4-348a-4168-94de-2423d4d57146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:21.412629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f2dc4-348a-4168-94de-2423d4d57146 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:21.413265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f2dc4-348a-4168-94de-2423d4d57146 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:21.415086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef9c89-8390-48ae-b1ff-ca8ee943ee7f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:21.415429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f2dc4-348a-4168-94de-2423d4d57146 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:21.434051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1120/2234] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:21 2022] PUT /identity/v3/projects/e278523cbcdd4d19af6f03bc633c9f78/users/fc525cd22ba84eca8d54b4a6ef4eb3cc/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:21.435625 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef9c89-8390-48ae-b1ff-ca8ee943ee7f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:21.456030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1115/2235] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:21 2022] POST /identity/v3/users => generated 289 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:21.456030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84189613-630a-47df-a338-820d961a84ee None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:21.456030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84189613-630a-47df-a338-820d961a84ee None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:21.456659 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84189613-630a-47df-a338-820d961a84ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:21.503780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-84189613-630a-47df-a338-820d961a84ee None None] MFA Rules not processed for user `fc525cd22ba84eca8d54b4a6ef4eb3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:21.505446 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-84189613-630a-47df-a338-820d961a84ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:21.544751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef9c89-8390-48ae-b1ff-ca8ee943ee7f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:21.553749 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd72012-95fc-4918-9a8b-2eafc1c90730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:21.554774 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd72012-95fc-4918-9a8b-2eafc1c90730 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:21.555243 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd72012-95fc-4918-9a8b-2eafc1c90730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:21.555704 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd72012-95fc-4918-9a8b-2eafc1c90730 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:21.556814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd72012-95fc-4918-9a8b-2eafc1c90730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:21.559825 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd72012-95fc-4918-9a8b-2eafc1c90730 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:21.583211 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1121/2236] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:21.584596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd72012-95fc-4918-9a8b-2eafc1c90730 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:21.599905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd72012-95fc-4918-9a8b-2eafc1c90730 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:21.607090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b8cdf-e0e8-47cd-8d7c-296834f56775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:21.608325 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b8cdf-e0e8-47cd-8d7c-296834f56775 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:21.608724 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b8cdf-e0e8-47cd-8d7c-296834f56775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:21.609353 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b8cdf-e0e8-47cd-8d7c-296834f56775 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:21.611109 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b8cdf-e0e8-47cd-8d7c-296834f56775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:21.613755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b8cdf-e0e8-47cd-8d7c-296834f56775 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:21.623532 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1116/2237] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:21.624565 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84189613-630a-47df-a338-820d961a84ee None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:21.637764 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84189613-630a-47df-a338-820d961a84ee None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:21.643009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1122/2238] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:21.643009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b8cdf-e0e8-47cd-8d7c-296834f56775 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:21.645984 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bc6b8cdf-e0e8-47cd-8d7c-296834f56775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:21.649282 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4783d57-a60f-4bef-8ba0-27c8fcbd4277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:21.650530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4783d57-a60f-4bef-8ba0-27c8fcbd4277 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:21.651191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4783d57-a60f-4bef-8ba0-27c8fcbd4277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:21.651679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4783d57-a60f-4bef-8ba0-27c8fcbd4277 admin admin] PATH_INFO: `/v3/projects/591ecc366fbe4f7992847dbba881ca88/users/38e2b00e11d24830965329e420d84e42/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:21.673310 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4783d57-a60f-4bef-8ba0-27c8fcbd4277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=591ecc366fbe4f7992847dbba881ca88, user_id=38e2b00e11d24830965329e420d84e42, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:21.676230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4783d57-a60f-4bef-8ba0-27c8fcbd4277 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:21.694644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1117/2239] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:21 2022] PUT /identity/v3/projects/591ecc366fbe4f7992847dbba881ca88/users/38e2b00e11d24830965329e420d84e42/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:21.695519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4783d57-a60f-4bef-8ba0-27c8fcbd4277 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:21.733974 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:21.764741 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b8cdf-e0e8-47cd-8d7c-296834f56775 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:21.789331 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4783d57-a60f-4bef-8ba0-27c8fcbd4277 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:21.798501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375a20f-b1b4-4092-8c0f-95695f9fdbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:21.799797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375a20f-b1b4-4092-8c0f-95695f9fdbb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:21.800301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375a20f-b1b4-4092-8c0f-95695f9fdbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:21.800797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375a20f-b1b4-4092-8c0f-95695f9fdbb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:21.802037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375a20f-b1b4-4092-8c0f-95695f9fdbb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:21.805711 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375a20f-b1b4-4092-8c0f-95695f9fdbb5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:21.827377 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1118/2240] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:21.829677 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495478a3-e19a-421b-a5cb-56b3f0cf87db None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:21.830117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495478a3-e19a-421b-a5cb-56b3f0cf87db None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:21.830551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495478a3-e19a-421b-a5cb-56b3f0cf87db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:21.847477 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119ef08-1f49-488b-b651-3bf5a0080ef8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:21.848446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119ef08-1f49-488b-b651-3bf5a0080ef8 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:21.848732 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119ef08-1f49-488b-b651-3bf5a0080ef8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:21.849005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119ef08-1f49-488b-b651-3bf5a0080ef8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:21.866196 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:21.889659 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-495478a3-e19a-421b-a5cb-56b3f0cf87db None None] MFA Rules not processed for user `38e2b00e11d24830965329e420d84e42`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:21.892014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-495478a3-e19a-421b-a5cb-56b3f0cf87db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:21.959099 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119ef08-1f49-488b-b651-3bf5a0080ef8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:21.961372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119ef08-1f49-488b-b651-3bf5a0080ef8 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:21.978229 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1123/2241] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:16:21.980314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119ef08-1f49-488b-b651-3bf5a0080ef8 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:21.997148 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119ef08-1f49-488b-b651-3bf5a0080ef8 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:22.018615 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb9dd4-20f4-449f-ac60-75679571a593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:22.019513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb9dd4-20f4-449f-ac60-75679571a593 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.019791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb9dd4-20f4-449f-ac60-75679571a593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.020065 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb9dd4-20f4-449f-ac60-75679571a593 admin admin] PATH_INFO: `/v3/projects/b52ac4a27fe84303970a9f7e104c5298/users/72cc5f5b1b6b4fb28d57c7ff7931fdff/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.027716 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1119/2242] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:22.048056 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495478a3-e19a-421b-a5cb-56b3f0cf87db None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:22.066257 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb9dd4-20f4-449f-ac60-75679571a593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b52ac4a27fe84303970a9f7e104c5298, user_id=72cc5f5b1b6b4fb28d57c7ff7931fdff, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:22.070054 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb9dd4-20f4-449f-ac60-75679571a593 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:22.088371 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495478a3-e19a-421b-a5cb-56b3f0cf87db None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:22.101848 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1124/2243] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:21 2022] PUT /identity/v3/projects/b52ac4a27fe84303970a9f7e104c5298/users/72cc5f5b1b6b4fb28d57c7ff7931fdff/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:22.107436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a36f2d-940d-4bcf-9927-682713b7c17a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.107726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a36f2d-940d-4bcf-9927-682713b7c17a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.108414 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a36f2d-940d-4bcf-9927-682713b7c17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.138874 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-52a36f2d-940d-4bcf-9927-682713b7c17a None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:22.138874 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-52a36f2d-940d-4bcf-9927-682713b7c17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:22.258344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1125/2244] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:22.260174 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a36f2d-940d-4bcf-9927-682713b7c17a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:22.263806 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3201b6-1ac3-40c1-8b82-ea2fb74b4d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:22.264450 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3201b6-1ac3-40c1-8b82-ea2fb74b4d88 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.264729 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3201b6-1ac3-40c1-8b82-ea2fb74b4d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.265002 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3201b6-1ac3-40c1-8b82-ea2fb74b4d88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.267379 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3201b6-1ac3-40c1-8b82-ea2fb74b4d88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:22.269061 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3201b6-1ac3-40c1-8b82-ea2fb74b4d88 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:22.333946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1120/2245] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:22 2022] POST /identity/v3/projects => generated 375 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:22.336386 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3201b6-1ac3-40c1-8b82-ea2fb74b4d88 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:22.340880 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6a3201b6-1ac3-40c1-8b82-ea2fb74b4d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:22.373011 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a36f2d-940d-4bcf-9927-682713b7c17a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:22.402162 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:22.431116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3201b6-1ac3-40c1-8b82-ea2fb74b4d88 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:22.437447 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f9324-9809-4cc7-a3e4-79e89e2cc5e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:22.438865 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f9324-9809-4cc7-a3e4-79e89e2cc5e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.439289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:22.439476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f9324-9809-4cc7-a3e4-79e89e2cc5e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.439736 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f9324-9809-4cc7-a3e4-79e89e2cc5e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.463510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7d416-e6ce-44cc-82a0-ffa8887ad572 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:22.464392 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7d416-e6ce-44cc-82a0-ffa8887ad572 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.464955 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7d416-e6ce-44cc-82a0-ffa8887ad572 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.466305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7d416-e6ce-44cc-82a0-ffa8887ad572 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.470720 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a8a7d416-e6ce-44cc-82a0-ffa8887ad572 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:22.547722 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f9324-9809-4cc7-a3e4-79e89e2cc5e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:22.551413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f9324-9809-4cc7-a3e4-79e89e2cc5e2 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:22.558014 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7d416-e6ce-44cc-82a0-ffa8887ad572 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:22.560127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7d416-e6ce-44cc-82a0-ffa8887ad572 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:22.568879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1121/2246] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:22.570515 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f9324-9809-4cc7-a3e4-79e89e2cc5e2 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:22.578469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1126/2247] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:22.583837 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f9324-9809-4cc7-a3e4-79e89e2cc5e2 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:22.594040 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf9a8d-9a22-4466-9da8-4e4ed0a4b599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:22.602553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf9a8d-9a22-4466-9da8-4e4ed0a4b599 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.602553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf9a8d-9a22-4466-9da8-4e4ed0a4b599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.602553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf9a8d-9a22-4466-9da8-4e4ed0a4b599 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.606501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf9a8d-9a22-4466-9da8-4e4ed0a4b599 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:22.611108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf9a8d-9a22-4466-9da8-4e4ed0a4b599 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:22.652252 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1122/2248] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:22 2022] POST /identity/v3/users => generated 294 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:22.659177 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7d416-e6ce-44cc-82a0-ffa8887ad572 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:22.671982 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7d416-e6ce-44cc-82a0-ffa8887ad572 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:22.681419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbfc77-fe32-4553-8e7b-48809badcb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:22.683033 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbfc77-fe32-4553-8e7b-48809badcb9a admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.683642 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbfc77-fe32-4553-8e7b-48809badcb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.684461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbfc77-fe32-4553-8e7b-48809badcb9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.685571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bbfc77-fe32-4553-8e7b-48809badcb9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:22.687623 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bbfc77-fe32-4553-8e7b-48809badcb9a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:22.707965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1127/2249] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:22.709263 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf9a8d-9a22-4466-9da8-4e4ed0a4b599 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:22.722872 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf9a8d-9a22-4466-9da8-4e4ed0a4b599 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:22.729220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5a3b9-622d-458a-8f4a-7871a24937e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:22.730325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5a3b9-622d-458a-8f4a-7871a24937e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.731045 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5a3b9-622d-458a-8f4a-7871a24937e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.731468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5a3b9-622d-458a-8f4a-7871a24937e1 admin admin] PATH_INFO: `/v3/projects/e15f3367eb4146768bf0e2c4efa81d5c/users/45a47586910e477381bee2b82a9eb425/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.755438 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5a3b9-622d-458a-8f4a-7871a24937e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e15f3367eb4146768bf0e2c4efa81d5c, user_id=45a47586910e477381bee2b82a9eb425, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:22.759859 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5a3b9-622d-458a-8f4a-7871a24937e1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:22.780363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1123/2250] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:22 2022] PUT /identity/v3/projects/e15f3367eb4146768bf0e2c4efa81d5c/users/45a47586910e477381bee2b82a9eb425/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:22.781435 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbfc77-fe32-4553-8e7b-48809badcb9a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:22.808075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbfc77-fe32-4553-8e7b-48809badcb9a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:22.991708 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149f1cb-c39c-4467-9ad2-8b098e60bd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:22.992445 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149f1cb-c39c-4467-9ad2-8b098e60bd8c admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:22.992854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149f1cb-c39c-4467-9ad2-8b098e60bd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:22.993248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149f1cb-c39c-4467-9ad2-8b098e60bd8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:22.995528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149f1cb-c39c-4467-9ad2-8b098e60bd8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:22.998040 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149f1cb-c39c-4467-9ad2-8b098e60bd8c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:23.015685 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1128/2251] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:16:23.021600 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5a3b9-622d-458a-8f4a-7871a24937e1 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:23.051645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5a3b9-622d-458a-8f4a-7871a24937e1 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:23.078695 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc7f3a-a405-496b-adf3-7aad61462fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:23.080906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc7f3a-a405-496b-adf3-7aad61462fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:23.081388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc7f3a-a405-496b-adf3-7aad61462fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:23.081724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc7f3a-a405-496b-adf3-7aad61462fe2 admin admin] PATH_INFO: `/v3/projects/e15f3367eb4146768bf0e2c4efa81d5c/users/45a47586910e477381bee2b82a9eb425/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:23.084116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc7f3a-a405-496b-adf3-7aad61462fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e15f3367eb4146768bf0e2c4efa81d5c, user_id=45a47586910e477381bee2b82a9eb425, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:23.089049 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc7f3a-a405-496b-adf3-7aad61462fe2 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:23.109678 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1124/2252] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:23 2022] PUT /identity/v3/projects/e15f3367eb4146768bf0e2c4efa81d5c/users/45a47586910e477381bee2b82a9eb425/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:23.120415 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ec4a5-bb8c-439e-8484-9db1c320733f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:23.120849 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ec4a5-bb8c-439e-8484-9db1c320733f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:23.121876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ec4a5-bb8c-439e-8484-9db1c320733f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:23.189400 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-214ec4a5-bb8c-439e-8484-9db1c320733f None None] MFA Rules not processed for user `45a47586910e477381bee2b82a9eb425`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:23.191432 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-214ec4a5-bb8c-439e-8484-9db1c320733f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:23.374847 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1129/2253] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:23.391100 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc7f3a-a405-496b-adf3-7aad61462fe2 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:23.393635 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-23bc7f3a-a405-496b-adf3-7aad61462fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:23.484331 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc7f3a-a405-496b-adf3-7aad61462fe2 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:23.488340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933e67c-8782-4a23-923b-d50b16f5f538 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:23.488971 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933e67c-8782-4a23-923b-d50b16f5f538 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:23.489530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933e67c-8782-4a23-923b-d50b16f5f538 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:23.489820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933e67c-8782-4a23-923b-d50b16f5f538 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:23.561054 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:23.580588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933e67c-8782-4a23-923b-d50b16f5f538 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:23.581851 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933e67c-8782-4a23-923b-d50b16f5f538 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:23.592920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1125/2254] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:27.638889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ec4a5-bb8c-439e-8484-9db1c320733f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:27.651037 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:27.665823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933e67c-8782-4a23-923b-d50b16f5f538 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:27.682368 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933e67c-8782-4a23-923b-d50b16f5f538 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:27.691710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227ec27-ad58-41da-bbca-5bb73592b36d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:27.694910 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227ec27-ad58-41da-bbca-5bb73592b36d service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:27.695495 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227ec27-ad58-41da-bbca-5bb73592b36d service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:27.696165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227ec27-ad58-41da-bbca-5bb73592b36d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:27.821462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ec4a5-bb8c-439e-8484-9db1c320733f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:27.835761 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4a337-c9bf-45b4-8b12-b24155bf752f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:27.836796 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4a337-c9bf-45b4-8b12-b24155bf752f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:27.837237 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4a337-c9bf-45b4-8b12-b24155bf752f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:27.837696 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4a337-c9bf-45b4-8b12-b24155bf752f admin admin] PATH_INFO: `/v3/users/0d4464700613439b9dc2b3864e67a695` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:27.839438 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4a337-c9bf-45b4-8b12-b24155bf752f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d4464700613439b9dc2b3864e67a695)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:27.841677 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4a337-c9bf-45b4-8b12-b24155bf752f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:27.896227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227ec27-ad58-41da-bbca-5bb73592b36d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:27.898311 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227ec27-ad58-41da-bbca-5bb73592b36d service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:27.934874 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1126/2255] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:27.935087 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-57c4a337-c9bf-45b4-8b12-b24155bf752f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d4464700613439b9dc2b3864e67a695'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:27.947896 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-57c4a337-c9bf-45b4-8b12-b24155bf752f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d4464700613439b9dc2b3864e67a695'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:27.969021 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-57c4a337-c9bf-45b4-8b12-b24155bf752f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d4464700613439b9dc2b3864e67a695'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:27.969962 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-57c4a337-c9bf-45b4-8b12-b24155bf752f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d4464700613439b9dc2b3864e67a695'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:27.984728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-57c4a337-c9bf-45b4-8b12-b24155bf752f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d4464700613439b9dc2b3864e67a695'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:27.997094 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1130/2256] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:27 2022] DELETE /identity/v3/users/0d4464700613439b9dc2b3864e67a695 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:28.003677 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d794d-b9c2-498f-8fe0-9916513d6026 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:28.004753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d794d-b9c2-498f-8fe0-9916513d6026 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:28.005832 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d794d-b9c2-498f-8fe0-9916513d6026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:28.067903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-881d794d-b9c2-498f-8fe0-9916513d6026 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:28.069619 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-881d794d-b9c2-498f-8fe0-9916513d6026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:28.201706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1127/2257] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:28.245989 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4a337-c9bf-45b4-8b12-b24155bf752f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:28.248184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-57c4a337-c9bf-45b4-8b12-b24155bf752f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:28.358538 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4a337-c9bf-45b4-8b12-b24155bf752f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:28.366189 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eed394-1dd9-44c1-ac30-bfdc0b926566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:28.368625 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eed394-1dd9-44c1-ac30-bfdc0b926566 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:28.369648 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eed394-1dd9-44c1-ac30-bfdc0b926566 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:28.369937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eed394-1dd9-44c1-ac30-bfdc0b926566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:28.385921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eed394-1dd9-44c1-ac30-bfdc0b926566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:28.387649 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eed394-1dd9-44c1-ac30-bfdc0b926566 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:28.415983 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1131/2258] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:29.278027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d794d-b9c2-498f-8fe0-9916513d6026 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:29.279535 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-881d794d-b9c2-498f-8fe0-9916513d6026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:29.380903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d794d-b9c2-498f-8fe0-9916513d6026 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:29.386145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d6278-25bf-4f57-9a60-7ef94c2f4f47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:29.388469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d6278-25bf-4f57-9a60-7ef94c2f4f47 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:29.388469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d6278-25bf-4f57-9a60-7ef94c2f4f47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:29.388843 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d6278-25bf-4f57-9a60-7ef94c2f4f47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:29.394822 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1d6d6278-25bf-4f57-9a60-7ef94c2f4f47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:29.477248 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:29.507492 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d6278-25bf-4f57-9a60-7ef94c2f4f47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:29.509077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d6278-25bf-4f57-9a60-7ef94c2f4f47 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:29.521665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eed394-1dd9-44c1-ac30-bfdc0b926566 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:29.523252 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-41eed394-1dd9-44c1-ac30-bfdc0b926566 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:29.533800 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1128/2259] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:29 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:29.543263 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eed394-1dd9-44c1-ac30-bfdc0b926566 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:29.551075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c2fd2-7bee-41a0-adf5-7d83ec3601f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:29.551761 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c2fd2-7bee-41a0-adf5-7d83ec3601f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:29.552048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c2fd2-7bee-41a0-adf5-7d83ec3601f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:29.552310 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c2fd2-7bee-41a0-adf5-7d83ec3601f3 admin admin] PATH_INFO: `/v3/projects/8550a73cb68548218b601f20fd8d1668` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:29.553611 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c2fd2-7bee-41a0-adf5-7d83ec3601f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8550a73cb68548218b601f20fd8d1668)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:29.559775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c2fd2-7bee-41a0-adf5-7d83ec3601f3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:29.618721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:29.683121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-807c2fd2-7bee-41a0-adf5-7d83ec3601f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8550a73cb68548218b601f20fd8d1668', 'request_id': 'req-807c2fd2-7bee-41a0-adf5-7d83ec3601f3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:29.693738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-807c2fd2-7bee-41a0-adf5-7d83ec3601f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8550a73cb68548218b601f20fd8d1668', 'request_id': 'req-807c2fd2-7bee-41a0-adf5-7d83ec3601f3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:29.703609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-807c2fd2-7bee-41a0-adf5-7d83ec3601f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8550a73cb68548218b601f20fd8d1668', 'request_id': 'req-807c2fd2-7bee-41a0-adf5-7d83ec3601f3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:29.726435 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-807c2fd2-7bee-41a0-adf5-7d83ec3601f3 admin admin] The token cache is being invalidate because project 8550a73cb68548218b601f20fd8d1668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:29.726818 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-807c2fd2-7bee-41a0-adf5-7d83ec3601f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:29.727965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-807c2fd2-7bee-41a0-adf5-7d83ec3601f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:29.731876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1132/2260] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:29 2022] DELETE /identity/v3/projects/8550a73cb68548218b601f20fd8d1668 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:29.739958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d6278-25bf-4f57-9a60-7ef94c2f4f47 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:29.762768 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d6278-25bf-4f57-9a60-7ef94c2f4f47 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:29.927140 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a43144-0129-4c8b-951a-e287080d9352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:29.928101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a43144-0129-4c8b-951a-e287080d9352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:29.928101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a43144-0129-4c8b-951a-e287080d9352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:29.928506 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a43144-0129-4c8b-951a-e287080d9352 admin admin] PATH_INFO: `/v3/users/fc525cd22ba84eca8d54b4a6ef4eb3cc` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:29.930278 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a43144-0129-4c8b-951a-e287080d9352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc525cd22ba84eca8d54b4a6ef4eb3cc)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:29.934061 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a43144-0129-4c8b-951a-e287080d9352 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:30.056159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d5a43144-0129-4c8b-951a-e287080d9352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc525cd22ba84eca8d54b4a6ef4eb3cc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:30.069738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d5a43144-0129-4c8b-951a-e287080d9352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc525cd22ba84eca8d54b4a6ef4eb3cc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:30.078437 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11bd40-1e0e-45b9-a282-bef81bf67be3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:30.080447 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11bd40-1e0e-45b9-a282-bef81bf67be3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:30.080640 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11bd40-1e0e-45b9-a282-bef81bf67be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:30.093674 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d5a43144-0129-4c8b-951a-e287080d9352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc525cd22ba84eca8d54b4a6ef4eb3cc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:30.095812 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d5a43144-0129-4c8b-951a-e287080d9352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc525cd22ba84eca8d54b4a6ef4eb3cc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:30.097081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d5a43144-0129-4c8b-951a-e287080d9352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc525cd22ba84eca8d54b4a6ef4eb3cc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:30.127440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1129/2261] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:29 2022] DELETE /identity/v3/users/fc525cd22ba84eca8d54b4a6ef4eb3cc => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:30.143350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-aa11bd40-1e0e-45b9-a282-bef81bf67be3 None None] MFA Rules not processed for user `72cc5f5b1b6b4fb28d57c7ff7931fdff`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:30.145958 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-aa11bd40-1e0e-45b9-a282-bef81bf67be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:30.275458 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1133/2262] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:30.293437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a43144-0129-4c8b-951a-e287080d9352 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:30.296151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d5a43144-0129-4c8b-951a-e287080d9352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:30.430632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:30.456757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11bd40-1e0e-45b9-a282-bef81bf67be3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:30.459806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-aa11bd40-1e0e-45b9-a282-bef81bf67be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:30.473946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a43144-0129-4c8b-951a-e287080d9352 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:30.486195 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd527c-80d4-4d13-b13e-767a9eb8b055 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:30.487195 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd527c-80d4-4d13-b13e-767a9eb8b055 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:30.487662 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd527c-80d4-4d13-b13e-767a9eb8b055 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:30.488108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd527c-80d4-4d13-b13e-767a9eb8b055 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:30.513978 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd527c-80d4-4d13-b13e-767a9eb8b055 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:30.514630 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd527c-80d4-4d13-b13e-767a9eb8b055 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:30.552245 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1130/2263] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:30.573275 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11bd40-1e0e-45b9-a282-bef81bf67be3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:30.578336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6669de-4a63-4244-9066-a0fa439af2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:30.579620 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6669de-4a63-4244-9066-a0fa439af2a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:30.579899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6669de-4a63-4244-9066-a0fa439af2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:30.580167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6669de-4a63-4244-9066-a0fa439af2a7 admin admin] PATH_INFO: `/v3/users/ade45b21943d4dd283dcdd1b450561d4` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:30.582339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6669de-4a63-4244-9066-a0fa439af2a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ade45b21943d4dd283dcdd1b450561d4)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:30.584848 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6669de-4a63-4244-9066-a0fa439af2a7 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:30.677948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:30.686004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3d6669de-4a63-4244-9066-a0fa439af2a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ade45b21943d4dd283dcdd1b450561d4'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:30.718426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3d6669de-4a63-4244-9066-a0fa439af2a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ade45b21943d4dd283dcdd1b450561d4'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:30.756997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3d6669de-4a63-4244-9066-a0fa439af2a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ade45b21943d4dd283dcdd1b450561d4'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:30.758965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3d6669de-4a63-4244-9066-a0fa439af2a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ade45b21943d4dd283dcdd1b450561d4'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:30.771661 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd527c-80d4-4d13-b13e-767a9eb8b055 service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:30.773386 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-aecd527c-80d4-4d13-b13e-767a9eb8b055 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:30.784984 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3d6669de-4a63-4244-9066-a0fa439af2a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ade45b21943d4dd283dcdd1b450561d4'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:30.807665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1134/2264] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:30 2022] DELETE /identity/v3/users/ade45b21943d4dd283dcdd1b450561d4 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:16:30.811990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd527c-80d4-4d13-b13e-767a9eb8b055 service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:30.814569 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98747c28-0c70-4f62-a1a9-6522b7a2ccc9 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:30.815069 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98747c28-0c70-4f62-a1a9-6522b7a2ccc9 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:30.815444 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98747c28-0c70-4f62-a1a9-6522b7a2ccc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:30.908992 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-98747c28-0c70-4f62-a1a9-6522b7a2ccc9 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:30.910542 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-98747c28-0c70-4f62-a1a9-6522b7a2ccc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:30.999399 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61030ab-4df4-4cba-95e1-ed2951bb011e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:31.000117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61030ab-4df4-4cba-95e1-ed2951bb011e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:31.000440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61030ab-4df4-4cba-95e1-ed2951bb011e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:31.001195 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61030ab-4df4-4cba-95e1-ed2951bb011e admin admin] PATH_INFO: `/v3/projects/e278523cbcdd4d19af6f03bc633c9f78` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:31.002936 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61030ab-4df4-4cba-95e1-ed2951bb011e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e278523cbcdd4d19af6f03bc633c9f78)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:31.005139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61030ab-4df4-4cba-95e1-ed2951bb011e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:31.112201 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1135/2265] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:31.152892 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98747c28-0c70-4f62-a1a9-6522b7a2ccc9 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:31.154807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-98747c28-0c70-4f62-a1a9-6522b7a2ccc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:31.166015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c61030ab-4df4-4cba-95e1-ed2951bb011e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e278523cbcdd4d19af6f03bc633c9f78', 'request_id': 'req-c61030ab-4df4-4cba-95e1-ed2951bb011e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:31.189434 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c61030ab-4df4-4cba-95e1-ed2951bb011e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e278523cbcdd4d19af6f03bc633c9f78', 'request_id': 'req-c61030ab-4df4-4cba-95e1-ed2951bb011e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:31.207074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c61030ab-4df4-4cba-95e1-ed2951bb011e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e278523cbcdd4d19af6f03bc633c9f78', 'request_id': 'req-c61030ab-4df4-4cba-95e1-ed2951bb011e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:31.232231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c61030ab-4df4-4cba-95e1-ed2951bb011e admin admin] The token cache is being invalidate because project e278523cbcdd4d19af6f03bc633c9f78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:31.233808 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c61030ab-4df4-4cba-95e1-ed2951bb011e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:31.236926 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c61030ab-4df4-4cba-95e1-ed2951bb011e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:31.239241 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1131/2266] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:30 2022] DELETE /identity/v3/projects/e278523cbcdd4d19af6f03bc633c9f78 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:31.248098 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33c78d-fad6-41ea-bca2-77384ebdafdc None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:31.248628 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33c78d-fad6-41ea-bca2-77384ebdafdc None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:31.249133 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33c78d-fad6-41ea-bca2-77384ebdafdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:31.327053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0a33c78d-fad6-41ea-bca2-77384ebdafdc None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:31.333766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0a33c78d-fad6-41ea-bca2-77384ebdafdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:31.406328 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98747c28-0c70-4f62-a1a9-6522b7a2ccc9 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:31.536652 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1132/2267] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:31.547264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd9130-2efe-4924-8fe7-7b805ba64974 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:31.548540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd9130-2efe-4924-8fe7-7b805ba64974 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:31.549340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd9130-2efe-4924-8fe7-7b805ba64974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:31.605839 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0bdd9130-2efe-4924-8fe7-7b805ba64974 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:31.607833 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3a5eb-ef13-467e-af87-9426ce1a6917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:31.608449 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0bdd9130-2efe-4924-8fe7-7b805ba64974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:31.608537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3a5eb-ef13-467e-af87-9426ce1a6917 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:31.609633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3a5eb-ef13-467e-af87-9426ce1a6917 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:31.610135 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3a5eb-ef13-467e-af87-9426ce1a6917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:31.619449 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9ce3a5eb-ef13-467e-af87-9426ce1a6917 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:31.639058 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1133/2268] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:31.640516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd9130-2efe-4924-8fe7-7b805ba64974 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:31.656581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3a5eb-ef13-467e-af87-9426ce1a6917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:31.658119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd9130-2efe-4924-8fe7-7b805ba64974 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:31.665214 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3a5eb-ef13-467e-af87-9426ce1a6917 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:31.671609 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c60bb-fb7b-45b6-bfbc-f37899be5f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:31.672261 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c60bb-fb7b-45b6-bfbc-f37899be5f62 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:31.672634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c60bb-fb7b-45b6-bfbc-f37899be5f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:31.672917 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c60bb-fb7b-45b6-bfbc-f37899be5f62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:31.675163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c60bb-fb7b-45b6-bfbc-f37899be5f62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:16:31.675519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c60bb-fb7b-45b6-bfbc-f37899be5f62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:31.680051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c60bb-fb7b-45b6-bfbc-f37899be5f62 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:31.685606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:31.712707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1134/2270] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:16:31.712969 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1136/2270] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:31.715808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3a5eb-ef13-467e-af87-9426ce1a6917 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:31.730389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3a5eb-ef13-467e-af87-9426ce1a6917 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:31.739257 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69aa79-2b04-4d6e-9913-491934b268a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:31.740145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69aa79-2b04-4d6e-9913-491934b268a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:31.740472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69aa79-2b04-4d6e-9913-491934b268a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:31.740838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69aa79-2b04-4d6e-9913-491934b268a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:31.741560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69aa79-2b04-4d6e-9913-491934b268a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:31.745579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69aa79-2b04-4d6e-9913-491934b268a4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:31.819041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:31.847047 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c60bb-fb7b-45b6-bfbc-f37899be5f62 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:31.847291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1137/2271] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:31 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:31.861472 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c60bb-fb7b-45b6-bfbc-f37899be5f62 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:31.942719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804eae0-5862-4120-93df-a11e13f07e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:31.943811 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804eae0-5862-4120-93df-a11e13f07e23 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:31.945081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804eae0-5862-4120-93df-a11e13f07e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:31.945598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804eae0-5862-4120-93df-a11e13f07e23 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:31.946428 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a804eae0-5862-4120-93df-a11e13f07e23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:31.949638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a804eae0-5862-4120-93df-a11e13f07e23 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:31.991953 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1135/2272] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:31 2022] POST /identity/v3/users => generated 288 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:31.995819 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69aa79-2b04-4d6e-9913-491934b268a4 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:32.011321 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69aa79-2b04-4d6e-9913-491934b268a4 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:32.018524 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400eecca-7903-4dd6-aba0-7dabb5c946ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:32.019587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400eecca-7903-4dd6-aba0-7dabb5c946ed admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.020067 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400eecca-7903-4dd6-aba0-7dabb5c946ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.020558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400eecca-7903-4dd6-aba0-7dabb5c946ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.021605 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400eecca-7903-4dd6-aba0-7dabb5c946ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:32.024310 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400eecca-7903-4dd6-aba0-7dabb5c946ed admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:32.037888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1138/2273] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:16:32.042946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804eae0-5862-4120-93df-a11e13f07e23 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:32.074739 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804eae0-5862-4120-93df-a11e13f07e23 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:32.083706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a704d-07ab-4cc8-9ec1-ac2ef89db9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:32.084346 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a704d-07ab-4cc8-9ec1-ac2ef89db9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.084626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a704d-07ab-4cc8-9ec1-ac2ef89db9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.084894 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a704d-07ab-4cc8-9ec1-ac2ef89db9c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.085570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a704d-07ab-4cc8-9ec1-ac2ef89db9c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:32.088018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a704d-07ab-4cc8-9ec1-ac2ef89db9c7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:32.103844 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1136/2274] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:16:32.107851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400eecca-7903-4dd6-aba0-7dabb5c946ed admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:32.123186 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400eecca-7903-4dd6-aba0-7dabb5c946ed admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:32.130069 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091c03b-5e00-41ec-80b9-b08fa30b33f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:32.131669 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091c03b-5e00-41ec-80b9-b08fa30b33f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.132315 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091c03b-5e00-41ec-80b9-b08fa30b33f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.132544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091c03b-5e00-41ec-80b9-b08fa30b33f3 admin admin] PATH_INFO: `/v3/projects/5f88c60a4a8745399943e21c85d92a1d/users/6ad4f18630864aa399bc274240078a75/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.154618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091c03b-5e00-41ec-80b9-b08fa30b33f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f88c60a4a8745399943e21c85d92a1d, user_id=6ad4f18630864aa399bc274240078a75, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:32.157944 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091c03b-5e00-41ec-80b9-b08fa30b33f3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:32.180405 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1139/2275] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:32 2022] PUT /identity/v3/projects/5f88c60a4a8745399943e21c85d92a1d/users/6ad4f18630864aa399bc274240078a75/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:32.191754 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31518363-9d54-49af-b3f8-5104ed0874f2 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.192199 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31518363-9d54-49af-b3f8-5104ed0874f2 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.192577 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31518363-9d54-49af-b3f8-5104ed0874f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.245946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-31518363-9d54-49af-b3f8-5104ed0874f2 None None] MFA Rules not processed for user `6ad4f18630864aa399bc274240078a75`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:32.245946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-31518363-9d54-49af-b3f8-5104ed0874f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:32.246575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091c03b-5e00-41ec-80b9-b08fa30b33f3 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:32.249223 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b091c03b-5e00-41ec-80b9-b08fa30b33f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:32.376745 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091c03b-5e00-41ec-80b9-b08fa30b33f3 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:32.378870 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1137/2276] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:16:32.388466 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4822d-e1a3-4fa6-b29c-846fbc880190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:32.389448 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4822d-e1a3-4fa6-b29c-846fbc880190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.389737 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4822d-e1a3-4fa6-b29c-846fbc880190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.390021 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4822d-e1a3-4fa6-b29c-846fbc880190 admin admin] PATH_INFO: `/v3/projects/ac737b893dca4bbeaadfa9112415bb4f` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.390625 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31518363-9d54-49af-b3f8-5104ed0874f2 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:32.392237 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4822d-e1a3-4fa6-b29c-846fbc880190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac737b893dca4bbeaadfa9112415bb4f)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:32.393308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-31518363-9d54-49af-b3f8-5104ed0874f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:32.394178 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4822d-e1a3-4fa6-b29c-846fbc880190 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:32.515655 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-84e4822d-e1a3-4fa6-b29c-846fbc880190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac737b893dca4bbeaadfa9112415bb4f', 'request_id': 'req-84e4822d-e1a3-4fa6-b29c-846fbc880190'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:32.530616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-84e4822d-e1a3-4fa6-b29c-846fbc880190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac737b893dca4bbeaadfa9112415bb4f', 'request_id': 'req-84e4822d-e1a3-4fa6-b29c-846fbc880190'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:32.541035 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31518363-9d54-49af-b3f8-5104ed0874f2 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:32.554440 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-84e4822d-e1a3-4fa6-b29c-846fbc880190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac737b893dca4bbeaadfa9112415bb4f', 'request_id': 'req-84e4822d-e1a3-4fa6-b29c-846fbc880190'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:32.574353 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:32.582721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-84e4822d-e1a3-4fa6-b29c-846fbc880190 admin admin] The token cache is being invalidate because project ac737b893dca4bbeaadfa9112415bb4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:32.583050 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-84e4822d-e1a3-4fa6-b29c-846fbc880190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:32.583925 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-84e4822d-e1a3-4fa6-b29c-846fbc880190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:32.589605 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1140/2277] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:32 2022] DELETE /identity/v3/projects/ac737b893dca4bbeaadfa9112415bb4f => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:32.591310 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea022fe-a788-45d8-bcb1-019580eee6b5 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.591599 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea022fe-a788-45d8-bcb1-019580eee6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.591872 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea022fe-a788-45d8-bcb1-019580eee6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.625984 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac3b8f-e1b3-4ecc-bc7d-acf03d383171 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50ef7db27744c7ebd2ec95cecc9d1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:32.627115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac3b8f-e1b3-4ecc-bc7d-acf03d383171 service glance] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.627738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-aea022fe-a788-45d8-bcb1-019580eee6b5 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:32.628525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac3b8f-e1b3-4ecc-bc7d-acf03d383171 service glance] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.629005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-aea022fe-a788-45d8-bcb1-019580eee6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:32.629296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac3b8f-e1b3-4ecc-bc7d-acf03d383171 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.632545 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-32ac3b8f-e1b3-4ecc-bc7d-acf03d383171 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:32.718852 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac3b8f-e1b3-4ecc-bc7d-acf03d383171 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:32.721483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:32.722737 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac3b8f-e1b3-4ecc-bc7d-acf03d383171 service glance] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:32.738697 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1138/2278] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:32.740152 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1141/2279] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:32.748781 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a7a65-7b82-4f5d-9cd1-65a735f158ba None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.749177 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a7a65-7b82-4f5d-9cd1-65a735f158ba None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.749480 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a7a65-7b82-4f5d-9cd1-65a735f158ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.777050 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-543a7a65-7b82-4f5d-9cd1-65a735f158ba None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:32.778591 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-543a7a65-7b82-4f5d-9cd1-65a735f158ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:32.789209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1139/2280] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:32.793216 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea022fe-a788-45d8-bcb1-019580eee6b5 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:32.804350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea022fe-a788-45d8-bcb1-019580eee6b5 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:32.832332 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822b9b3-1167-4e6b-81c0-acf01d72ac41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:32.832986 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822b9b3-1167-4e6b-81c0-acf01d72ac41 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.833265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822b9b3-1167-4e6b-81c0-acf01d72ac41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.833539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822b9b3-1167-4e6b-81c0-acf01d72ac41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.834368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822b9b3-1167-4e6b-81c0-acf01d72ac41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:16:32.834767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822b9b3-1167-4e6b-81c0-acf01d72ac41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:32.836409 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822b9b3-1167-4e6b-81c0-acf01d72ac41 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:32.862989 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1142/2281] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:16:32.865397 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a7a65-7b82-4f5d-9cd1-65a735f158ba None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:32.880925 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a7a65-7b82-4f5d-9cd1-65a735f158ba None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:32.886698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4da16-9a41-479e-a983-6ec240c56643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:32.887405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4da16-9a41-479e-a983-6ec240c56643 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:32.887691 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4da16-9a41-479e-a983-6ec240c56643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:32.887965 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4da16-9a41-479e-a983-6ec240c56643 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:32.888770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4da16-9a41-479e-a983-6ec240c56643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:32.890939 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4da16-9a41-479e-a983-6ec240c56643 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:32.962456 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1140/2282] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:32 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:32.966567 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822b9b3-1167-4e6b-81c0-acf01d72ac41 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:32.993414 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822b9b3-1167-4e6b-81c0-acf01d72ac41 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:33.097760 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d0b88-438e-440b-81ca-d40aa960091c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:33.098373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d0b88-438e-440b-81ca-d40aa960091c admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:33.098868 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d0b88-438e-440b-81ca-d40aa960091c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:33.104615 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d0b88-438e-440b-81ca-d40aa960091c admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:33.105549 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d0b88-438e-440b-81ca-d40aa960091c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:33.107597 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d0b88-438e-440b-81ca-d40aa960091c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:33.141786 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1143/2283] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:32 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:33.143193 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4da16-9a41-479e-a983-6ec240c56643 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:33.164850 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4da16-9a41-479e-a983-6ec240c56643 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:33.173087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04ec7e-9d17-457c-b5d5-a188e5b43f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:33.173713 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04ec7e-9d17-457c-b5d5-a188e5b43f2f admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:33.173989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04ec7e-9d17-457c-b5d5-a188e5b43f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:33.174259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04ec7e-9d17-457c-b5d5-a188e5b43f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:33.174962 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c04ec7e-9d17-457c-b5d5-a188e5b43f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:33.187056 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c04ec7e-9d17-457c-b5d5-a188e5b43f2f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:33.225011 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1141/2284] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:33.231502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d0b88-438e-440b-81ca-d40aa960091c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:33.260158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d0b88-438e-440b-81ca-d40aa960091c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:33.274297 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c6c61-da86-46f2-b378-7493781a92da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:33.278215 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c6c61-da86-46f2-b378-7493781a92da admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:33.278215 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c6c61-da86-46f2-b378-7493781a92da admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:33.278215 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c6c61-da86-46f2-b378-7493781a92da admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:33.278215 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c6c61-da86-46f2-b378-7493781a92da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:33.279780 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c6c61-da86-46f2-b378-7493781a92da admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:33.318504 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1144/2285] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:33.320329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04ec7e-9d17-457c-b5d5-a188e5b43f2f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:33.348128 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04ec7e-9d17-457c-b5d5-a188e5b43f2f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:33.356741 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165715ed-c069-4187-9662-a5a350c9b5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:33.358830 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165715ed-c069-4187-9662-a5a350c9b5ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:33.359300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165715ed-c069-4187-9662-a5a350c9b5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:33.360710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165715ed-c069-4187-9662-a5a350c9b5ce admin admin] PATH_INFO: `/v3/projects/fbc78793ec5c47aa9b56c6cbe6c4d9dc/users/ac26eb671f1a4a889e8d9c7c35a70019/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:33.398737 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165715ed-c069-4187-9662-a5a350c9b5ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc78793ec5c47aa9b56c6cbe6c4d9dc, user_id=ac26eb671f1a4a889e8d9c7c35a70019, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:33.403801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165715ed-c069-4187-9662-a5a350c9b5ce admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:33.427372 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1142/2286] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:33 2022] PUT /identity/v3/projects/fbc78793ec5c47aa9b56c6cbe6c4d9dc/users/ac26eb671f1a4a889e8d9c7c35a70019/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:16:33.430497 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348994c-a538-40dd-b80a-5dce6043ac2a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:33.431100 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348994c-a538-40dd-b80a-5dce6043ac2a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:33.431807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348994c-a538-40dd-b80a-5dce6043ac2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:33.503322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8348994c-a538-40dd-b80a-5dce6043ac2a None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:33.504856 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8348994c-a538-40dd-b80a-5dce6043ac2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:33.741229 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1145/2287] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:33.755212 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165715ed-c069-4187-9662-a5a350c9b5ce admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:33.759566 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-165715ed-c069-4187-9662-a5a350c9b5ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:33.784058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62780882-43f5-4664-a316-6419c429ebe5 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:33.786075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62780882-43f5-4664-a316-6419c429ebe5 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:33.786584 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62780882-43f5-4664-a316-6419c429ebe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:33.852650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-62780882-43f5-4664-a316-6419c429ebe5 None None] MFA Rules not processed for user `72cc5f5b1b6b4fb28d57c7ff7931fdff`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:33.855070 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-62780882-43f5-4664-a316-6419c429ebe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:33.950818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165715ed-c069-4187-9662-a5a350c9b5ce admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:33.956254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cc580-405c-4308-a546-a1185ed8305c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:33.957941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cc580-405c-4308-a546-a1185ed8305c service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:33.958438 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cc580-405c-4308-a546-a1185ed8305c service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:33.958748 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cc580-405c-4308-a546-a1185ed8305c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:33.981613 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99cc580-405c-4308-a546-a1185ed8305c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:33.981613 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99cc580-405c-4308-a546-a1185ed8305c service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:33.998208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1146/2288] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:34.023278 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1143/2289] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:37.051365 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62780882-43f5-4664-a316-6419c429ebe5 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:37.054838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-62780882-43f5-4664-a316-6419c429ebe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:37.082603 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:37.102822 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62780882-43f5-4664-a316-6419c429ebe5 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:37.109659 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c0aac-985a-44d5-a17d-fad1ea4aa088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:37.111257 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c0aac-985a-44d5-a17d-fad1ea4aa088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:37.111577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c0aac-985a-44d5-a17d-fad1ea4aa088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:37.111856 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c0aac-985a-44d5-a17d-fad1ea4aa088 admin admin] PATH_INFO: `/v3/users/08e5d528da0f4e89a1882d64b43ecd2b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:37.113173 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c0aac-985a-44d5-a17d-fad1ea4aa088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08e5d528da0f4e89a1882d64b43ecd2b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:37.116833 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c0aac-985a-44d5-a17d-fad1ea4aa088 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:37.223348 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-926c0aac-985a-44d5-a17d-fad1ea4aa088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08e5d528da0f4e89a1882d64b43ecd2b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:37.235759 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-926c0aac-985a-44d5-a17d-fad1ea4aa088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08e5d528da0f4e89a1882d64b43ecd2b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:37.257626 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-926c0aac-985a-44d5-a17d-fad1ea4aa088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08e5d528da0f4e89a1882d64b43ecd2b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:37.259459 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-926c0aac-985a-44d5-a17d-fad1ea4aa088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08e5d528da0f4e89a1882d64b43ecd2b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:37.283584 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-926c0aac-985a-44d5-a17d-fad1ea4aa088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08e5d528da0f4e89a1882d64b43ecd2b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:37.294550 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1147/2290] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:37 2022] DELETE /identity/v3/users/08e5d528da0f4e89a1882d64b43ecd2b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:37.296228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2921e45-6005-48fe-b911-42d8edc70050 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:37.296778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2921e45-6005-48fe-b911-42d8edc70050 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:37.296778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2921e45-6005-48fe-b911-42d8edc70050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:37.331133 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f2921e45-6005-48fe-b911-42d8edc70050 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:37.333210 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f2921e45-6005-48fe-b911-42d8edc70050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:37.343522 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:37.445529 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1144/2291] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:37.453267 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c0aac-985a-44d5-a17d-fad1ea4aa088 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:37.456274 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-926c0aac-985a-44d5-a17d-fad1ea4aa088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:37.555532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c0aac-985a-44d5-a17d-fad1ea4aa088 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:37.561729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be476be6-6cad-4255-ae06-3f55f29280c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:37.562454 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be476be6-6cad-4255-ae06-3f55f29280c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:37.563720 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be476be6-6cad-4255-ae06-3f55f29280c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:37.564289 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be476be6-6cad-4255-ae06-3f55f29280c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:37.575707 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be476be6-6cad-4255-ae06-3f55f29280c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:37.577102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be476be6-6cad-4255-ae06-3f55f29280c5 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:37.590558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1148/2292] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:16:38.747250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2921e45-6005-48fe-b911-42d8edc70050 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:38.751094 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f2921e45-6005-48fe-b911-42d8edc70050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:38.813697 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2921e45-6005-48fe-b911-42d8edc70050 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:38.849933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce233a33-4992-4ef8-ac33-ea3784526529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:38.864808 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce233a33-4992-4ef8-ac33-ea3784526529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:38.865692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce233a33-4992-4ef8-ac33-ea3784526529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:38.866433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce233a33-4992-4ef8-ac33-ea3784526529 admin admin] PATH_INFO: `/v3/projects/d06bd5bc00e54c3b97356760a3f5fa01` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:38.870692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce233a33-4992-4ef8-ac33-ea3784526529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d06bd5bc00e54c3b97356760a3f5fa01)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:38.885736 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce233a33-4992-4ef8-ac33-ea3784526529 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:39.021801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce233a33-4992-4ef8-ac33-ea3784526529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd06bd5bc00e54c3b97356760a3f5fa01', 'request_id': 'req-ce233a33-4992-4ef8-ac33-ea3784526529'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:39.042966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce233a33-4992-4ef8-ac33-ea3784526529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd06bd5bc00e54c3b97356760a3f5fa01', 'request_id': 'req-ce233a33-4992-4ef8-ac33-ea3784526529'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:39.060734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce233a33-4992-4ef8-ac33-ea3784526529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd06bd5bc00e54c3b97356760a3f5fa01', 'request_id': 'req-ce233a33-4992-4ef8-ac33-ea3784526529'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:39.105766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce233a33-4992-4ef8-ac33-ea3784526529 admin admin] The token cache is being invalidate because project d06bd5bc00e54c3b97356760a3f5fa01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:39.105766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce233a33-4992-4ef8-ac33-ea3784526529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:39.111131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce233a33-4992-4ef8-ac33-ea3784526529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:39.115738 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1145/2293] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:38 2022] DELETE /identity/v3/projects/d06bd5bc00e54c3b97356760a3f5fa01 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:39.117108 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be476be6-6cad-4255-ae06-3f55f29280c5 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:39.134980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be476be6-6cad-4255-ae06-3f55f29280c5 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:39.370370 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff1f08-ef16-4c39-8aeb-976f8623b6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:39.371411 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff1f08-ef16-4c39-8aeb-976f8623b6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:39.371885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff1f08-ef16-4c39-8aeb-976f8623b6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:39.372312 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff1f08-ef16-4c39-8aeb-976f8623b6f5 admin admin] PATH_INFO: `/v3/users/38e2b00e11d24830965329e420d84e42` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:39.373693 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff1f08-ef16-4c39-8aeb-976f8623b6f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38e2b00e11d24830965329e420d84e42)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:39.378506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff1f08-ef16-4c39-8aeb-976f8623b6f5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:39.458983 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:39.508249 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e6ff1f08-ef16-4c39-8aeb-976f8623b6f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38e2b00e11d24830965329e420d84e42'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:39.519294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e6ff1f08-ef16-4c39-8aeb-976f8623b6f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38e2b00e11d24830965329e420d84e42'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:39.541800 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e6ff1f08-ef16-4c39-8aeb-976f8623b6f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38e2b00e11d24830965329e420d84e42'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:39.542632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e6ff1f08-ef16-4c39-8aeb-976f8623b6f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38e2b00e11d24830965329e420d84e42'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:39.567604 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e6ff1f08-ef16-4c39-8aeb-976f8623b6f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38e2b00e11d24830965329e420d84e42'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:39.585368 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1149/2294] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:39 2022] DELETE /identity/v3/users/38e2b00e11d24830965329e420d84e42 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:40.447035 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce233a33-4992-4ef8-ac33-ea3784526529 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:40.448023 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ce233a33-4992-4ef8-ac33-ea3784526529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:40.478147 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce233a33-4992-4ef8-ac33-ea3784526529 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:40.706963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b5352-b22f-4fa9-8a43-ccdae6f7e252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:40.708858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b5352-b22f-4fa9-8a43-ccdae6f7e252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:40.709651 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b5352-b22f-4fa9-8a43-ccdae6f7e252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:40.711124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b5352-b22f-4fa9-8a43-ccdae6f7e252 admin admin] PATH_INFO: `/v3/projects/591ecc366fbe4f7992847dbba881ca88` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:40.714104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b5352-b22f-4fa9-8a43-ccdae6f7e252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=591ecc366fbe4f7992847dbba881ca88)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:40.720710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b5352-b22f-4fa9-8a43-ccdae6f7e252 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:40.859636 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:40.882315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cb3b5352-b22f-4fa9-8a43-ccdae6f7e252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '591ecc366fbe4f7992847dbba881ca88', 'request_id': 'req-cb3b5352-b22f-4fa9-8a43-ccdae6f7e252'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:40.896484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cb3b5352-b22f-4fa9-8a43-ccdae6f7e252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '591ecc366fbe4f7992847dbba881ca88', 'request_id': 'req-cb3b5352-b22f-4fa9-8a43-ccdae6f7e252'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:40.905147 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cb3b5352-b22f-4fa9-8a43-ccdae6f7e252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '591ecc366fbe4f7992847dbba881ca88', 'request_id': 'req-cb3b5352-b22f-4fa9-8a43-ccdae6f7e252'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:40.941247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cb3b5352-b22f-4fa9-8a43-ccdae6f7e252 admin admin] The token cache is being invalidate because project 591ecc366fbe4f7992847dbba881ca88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:40.941696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cb3b5352-b22f-4fa9-8a43-ccdae6f7e252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:40.942930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cb3b5352-b22f-4fa9-8a43-ccdae6f7e252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:40.945065 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1146/2295] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:40 2022] DELETE /identity/v3/projects/591ecc366fbe4f7992847dbba881ca88 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:40.950943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff1f08-ef16-4c39-8aeb-976f8623b6f5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:40.955243 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e6ff1f08-ef16-4c39-8aeb-976f8623b6f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:40.993506 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff1f08-ef16-4c39-8aeb-976f8623b6f5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:41.145894 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ab75a-2390-42fc-8002-26b1de6a1d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:41.148895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ab75a-2390-42fc-8002-26b1de6a1d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:41.149445 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ab75a-2390-42fc-8002-26b1de6a1d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:41.151041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ab75a-2390-42fc-8002-26b1de6a1d98 admin admin] PATH_INFO: `/v3/users/45a47586910e477381bee2b82a9eb425` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:41.159301 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ab75a-2390-42fc-8002-26b1de6a1d98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45a47586910e477381bee2b82a9eb425)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:41.161695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ab75a-2390-42fc-8002-26b1de6a1d98 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:41.436804 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-de1ab75a-2390-42fc-8002-26b1de6a1d98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45a47586910e477381bee2b82a9eb425'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:41.450490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-de1ab75a-2390-42fc-8002-26b1de6a1d98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45a47586910e477381bee2b82a9eb425'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:41.467752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-de1ab75a-2390-42fc-8002-26b1de6a1d98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45a47586910e477381bee2b82a9eb425'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:41.468946 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-de1ab75a-2390-42fc-8002-26b1de6a1d98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45a47586910e477381bee2b82a9eb425'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:41.483486 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-de1ab75a-2390-42fc-8002-26b1de6a1d98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45a47586910e477381bee2b82a9eb425'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:41.506450 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1150/2296] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:16:40 2022] DELETE /identity/v3/users/45a47586910e477381bee2b82a9eb425 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:42.874546 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b5352-b22f-4fa9-8a43-ccdae6f7e252 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:42.878570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cb3b5352-b22f-4fa9-8a43-ccdae6f7e252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:42.919242 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b5352-b22f-4fa9-8a43-ccdae6f7e252 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:43.101725 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb88678-eb9a-43d9-932c-e07e4eab74f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:43.102646 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb88678-eb9a-43d9-932c-e07e4eab74f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:43.103133 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb88678-eb9a-43d9-932c-e07e4eab74f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:43.103570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb88678-eb9a-43d9-932c-e07e4eab74f8 admin admin] PATH_INFO: `/v3/projects/e15f3367eb4146768bf0e2c4efa81d5c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:43.106018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb88678-eb9a-43d9-932c-e07e4eab74f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e15f3367eb4146768bf0e2c4efa81d5c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:43.110754 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb88678-eb9a-43d9-932c-e07e4eab74f8 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:43.230438 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2cb88678-eb9a-43d9-932c-e07e4eab74f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e15f3367eb4146768bf0e2c4efa81d5c', 'request_id': 'req-2cb88678-eb9a-43d9-932c-e07e4eab74f8'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:43.277168 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2cb88678-eb9a-43d9-932c-e07e4eab74f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e15f3367eb4146768bf0e2c4efa81d5c', 'request_id': 'req-2cb88678-eb9a-43d9-932c-e07e4eab74f8'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:43.306166 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2cb88678-eb9a-43d9-932c-e07e4eab74f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e15f3367eb4146768bf0e2c4efa81d5c', 'request_id': 'req-2cb88678-eb9a-43d9-932c-e07e4eab74f8'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:43.359658 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2cb88678-eb9a-43d9-932c-e07e4eab74f8 admin admin] The token cache is being invalidate because project e15f3367eb4146768bf0e2c4efa81d5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:16:43.360559 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2cb88678-eb9a-43d9-932c-e07e4eab74f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:43.361429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2cb88678-eb9a-43d9-932c-e07e4eab74f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:16:43.364578 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1147/2297] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:16:42 2022] DELETE /identity/v3/projects/e15f3367eb4146768bf0e2c4efa81d5c => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:43.371381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e88613-4b0a-4ba3-82e7-7e0a373ed989 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:43.371844 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e88613-4b0a-4ba3-82e7-7e0a373ed989 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:43.372407 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e88613-4b0a-4ba3-82e7-7e0a373ed989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:43.413248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-81e88613-4b0a-4ba3-82e7-7e0a373ed989 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:43.416145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-81e88613-4b0a-4ba3-82e7-7e0a373ed989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:43.493656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:43.677243 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1151/2298] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:43.685129 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617a0b4-5a3d-42e5-8668-d0d2aa9de7a7 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:43.685489 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617a0b4-5a3d-42e5-8668-d0d2aa9de7a7 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:43.685681 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617a0b4-5a3d-42e5-8668-d0d2aa9de7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:43.736970 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e617a0b4-5a3d-42e5-8668-d0d2aa9de7a7 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:43.738469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e617a0b4-5a3d-42e5-8668-d0d2aa9de7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:43.750444 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1148/2299] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:43.755283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e88613-4b0a-4ba3-82e7-7e0a373ed989 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:43.771878 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e88613-4b0a-4ba3-82e7-7e0a373ed989 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:43.791318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ec381-cb30-4e2b-986f-18722a5a46b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:43.791318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ec381-cb30-4e2b-986f-18722a5a46b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:43.791318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ec381-cb30-4e2b-986f-18722a5a46b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:43.791318 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ec381-cb30-4e2b-986f-18722a5a46b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:43.791900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ec381-cb30-4e2b-986f-18722a5a46b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:16:43.791900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ec381-cb30-4e2b-986f-18722a5a46b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:43.791900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ec381-cb30-4e2b-986f-18722a5a46b5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:43.834206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1152/2300] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:16:43.841605 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617a0b4-5a3d-42e5-8668-d0d2aa9de7a7 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:43.865376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617a0b4-5a3d-42e5-8668-d0d2aa9de7a7 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:43.872033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf11962-3d47-4508-be1e-f3d6d2f48fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:43.872765 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf11962-3d47-4508-be1e-f3d6d2f48fc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:43.873173 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf11962-3d47-4508-be1e-f3d6d2f48fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:43.873607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf11962-3d47-4508-be1e-f3d6d2f48fc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:43.874523 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf11962-3d47-4508-be1e-f3d6d2f48fc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:43.881326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf11962-3d47-4508-be1e-f3d6d2f48fc3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:43.955457 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ec381-cb30-4e2b-986f-18722a5a46b5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:43.956206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1149/2301] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:43 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:43.984199 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ec381-cb30-4e2b-986f-18722a5a46b5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:44.031693 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2de606-d5a9-4ce5-99f6-f33f11a996f3 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:44.032627 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2de606-d5a9-4ce5-99f6-f33f11a996f3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:44.032839 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2de606-d5a9-4ce5-99f6-f33f11a996f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:44.070145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-6f2de606-d5a9-4ce5-99f6-f33f11a996f3 None None] MFA Rules not processed for user `ac26eb671f1a4a889e8d9c7c35a70019`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:44.071443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6f2de606-d5a9-4ce5-99f6-f33f11a996f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:44.086745 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:44.126866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a06bf-7d9d-4cd8-8a3c-6f2b1367b969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:44.128075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a06bf-7d9d-4cd8-8a3c-6f2b1367b969 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:44.128540 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a06bf-7d9d-4cd8-8a3c-6f2b1367b969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:44.129023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a06bf-7d9d-4cd8-8a3c-6f2b1367b969 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:44.130393 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a06bf-7d9d-4cd8-8a3c-6f2b1367b969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:44.138639 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a06bf-7d9d-4cd8-8a3c-6f2b1367b969 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:44.265497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1150/2302] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:44.271216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2de606-d5a9-4ce5-99f6-f33f11a996f3 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:44.274562 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6f2de606-d5a9-4ce5-99f6-f33f11a996f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:44.288034 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1153/2303] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:43 2022] POST /identity/v3/users => generated 283 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:16:44.293882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a06bf-7d9d-4cd8-8a3c-6f2b1367b969 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:44.305908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2de606-d5a9-4ce5-99f6-f33f11a996f3 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:44.313473 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a06bf-7d9d-4cd8-8a3c-6f2b1367b969 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:44.321127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056323d-2ec7-413a-94df-96fc3902fe9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:44.321849 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056323d-2ec7-413a-94df-96fc3902fe9a admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:44.322221 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056323d-2ec7-413a-94df-96fc3902fe9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:44.322565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056323d-2ec7-413a-94df-96fc3902fe9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:44.324133 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056323d-2ec7-413a-94df-96fc3902fe9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:44.324502 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f8d92-3611-4f37-9d0a-3a278aa4c4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:44.326141 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f8d92-3611-4f37-9d0a-3a278aa4c4fa admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:44.326816 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f8d92-3611-4f37-9d0a-3a278aa4c4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:44.327010 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056323d-2ec7-413a-94df-96fc3902fe9a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:44.327296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f8d92-3611-4f37-9d0a-3a278aa4c4fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:44.328964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f8d92-3611-4f37-9d0a-3a278aa4c4fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:44.333628 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f8d92-3611-4f37-9d0a-3a278aa4c4fa admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:44.345479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1154/2304] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:16:44.347115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056323d-2ec7-413a-94df-96fc3902fe9a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:44.363593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056323d-2ec7-413a-94df-96fc3902fe9a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:44.370385 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4c6c9-ce30-464d-9a04-38d6151c1172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:44.371320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4c6c9-ce30-464d-9a04-38d6151c1172 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:44.371927 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4c6c9-ce30-464d-9a04-38d6151c1172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:44.382812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4c6c9-ce30-464d-9a04-38d6151c1172 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:44.384048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4c6c9-ce30-464d-9a04-38d6151c1172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:44.386493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4c6c9-ce30-464d-9a04-38d6151c1172 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:44.409158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1155/2305] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:44.410535 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4c6c9-ce30-464d-9a04-38d6151c1172 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:44.424298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4c6c9-ce30-464d-9a04-38d6151c1172 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:44.432366 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8459c8-f3c0-46ed-993e-8fec2e014f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:44.433813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8459c8-f3c0-46ed-993e-8fec2e014f01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:44.434121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8459c8-f3c0-46ed-993e-8fec2e014f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:44.434834 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8459c8-f3c0-46ed-993e-8fec2e014f01 admin admin] PATH_INFO: `/v3/projects/86604c9fb44c467098f915ff13b020f2/users/6077450b6a05477fb3eb3af7ad3889bc/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:44.457710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1151/2306] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:16:44 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:16:44.463604 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f8d92-3611-4f37-9d0a-3a278aa4c4fa admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:44.482902 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8459c8-f3c0-46ed-993e-8fec2e014f01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86604c9fb44c467098f915ff13b020f2, user_id=6077450b6a05477fb3eb3af7ad3889bc, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:44.491945 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8459c8-f3c0-46ed-993e-8fec2e014f01 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:44.523258 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1156/2307] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:44 2022] PUT /identity/v3/projects/86604c9fb44c467098f915ff13b020f2/users/6077450b6a05477fb3eb3af7ad3889bc/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:44.545983 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2128b-fe96-4f3b-9a08-59a01fd1edd0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:44.546321 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2128b-fe96-4f3b-9a08-59a01fd1edd0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:44.547146 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2128b-fe96-4f3b-9a08-59a01fd1edd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:44.629846 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f8d92-3611-4f37-9d0a-3a278aa4c4fa admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:44.636005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-07a2128b-fe96-4f3b-9a08-59a01fd1edd0 None None] MFA Rules not processed for user `6077450b6a05477fb3eb3af7ad3889bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:44.637756 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-07a2128b-fe96-4f3b-9a08-59a01fd1edd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:44.706411 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:44.730833 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4eaa7d-989e-4a9c-9ed7-6034673cf354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:44.731470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4eaa7d-989e-4a9c-9ed7-6034673cf354 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:44.731750 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4eaa7d-989e-4a9c-9ed7-6034673cf354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:44.732024 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4eaa7d-989e-4a9c-9ed7-6034673cf354 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:44.732675 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4eaa7d-989e-4a9c-9ed7-6034673cf354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:44.736405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4eaa7d-989e-4a9c-9ed7-6034673cf354 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:44.768226 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1157/2308] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:44.778512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1152/2309] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:16:44 2022] POST /identity/v3/users => generated 299 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:16:44.783026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2128b-fe96-4f3b-9a08-59a01fd1edd0 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:44.785012 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-07a2128b-fe96-4f3b-9a08-59a01fd1edd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:44.787959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4eaa7d-989e-4a9c-9ed7-6034673cf354 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:44.804747 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4eaa7d-989e-4a9c-9ed7-6034673cf354 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:44.810309 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabb0ea-4c7b-48dc-8784-12953a29eb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:44.811040 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabb0ea-4c7b-48dc-8784-12953a29eb9e admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:44.811331 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabb0ea-4c7b-48dc-8784-12953a29eb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:44.811874 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabb0ea-4c7b-48dc-8784-12953a29eb9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:44.812559 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabb0ea-4c7b-48dc-8784-12953a29eb9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:44.815720 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabb0ea-4c7b-48dc-8784-12953a29eb9e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:44.833437 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1153/2310] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:16:44.839216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabb0ea-4c7b-48dc-8784-12953a29eb9e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:44.852007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabb0ea-4c7b-48dc-8784-12953a29eb9e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:44.861250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d35cce-c85f-4458-8b87-4970a886eaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:44.862459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d35cce-c85f-4458-8b87-4970a886eaa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:44.862845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d35cce-c85f-4458-8b87-4970a886eaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:44.863239 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d35cce-c85f-4458-8b87-4970a886eaa5 admin admin] PATH_INFO: `/v3/projects/613022bf2be14f1aa7f842a6eb63cb35/users/ad292b1a14684657be07ac57f8ae0ad5/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:44.898781 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d35cce-c85f-4458-8b87-4970a886eaa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=613022bf2be14f1aa7f842a6eb63cb35, user_id=ad292b1a14684657be07ac57f8ae0ad5, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:44.898781 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d35cce-c85f-4458-8b87-4970a886eaa5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:44.903339 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:44.920140 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2128b-fe96-4f3b-9a08-59a01fd1edd0 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:44.925828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1154/2311] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:44 2022] PUT /identity/v3/projects/613022bf2be14f1aa7f842a6eb63cb35/users/ad292b1a14684657be07ac57f8ae0ad5/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:44.929807 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d35cce-c85f-4458-8b87-4970a886eaa5 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:44.999141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c66c4-79ed-4953-97d0-a4cd26872c02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:44.999640 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c66c4-79ed-4953-97d0-a4cd26872c02 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:44.999927 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c66c4-79ed-4953-97d0-a4cd26872c02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:45.000192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c66c4-79ed-4953-97d0-a4cd26872c02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:45.045269 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d35cce-c85f-4458-8b87-4970a886eaa5 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:45.051063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8ad77-d032-4dcd-9d37-8e12431472a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:45.051975 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8ad77-d032-4dcd-9d37-8e12431472a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:45.052398 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8ad77-d032-4dcd-9d37-8e12431472a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:45.052775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8ad77-d032-4dcd-9d37-8e12431472a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:45.053667 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8ad77-d032-4dcd-9d37-8e12431472a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:45.055533 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8ad77-d032-4dcd-9d37-8e12431472a5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:45.069086 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1155/2312] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:16:45.073674 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8ad77-d032-4dcd-9d37-8e12431472a5 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:45.086448 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8ad77-d032-4dcd-9d37-8e12431472a5 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:45.093027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce40b5d-62be-4cda-865c-8c2bea0ff1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:45.093825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce40b5d-62be-4cda-865c-8c2bea0ff1d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:45.094233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce40b5d-62be-4cda-865c-8c2bea0ff1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:45.094617 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce40b5d-62be-4cda-865c-8c2bea0ff1d7 admin admin] PATH_INFO: `/v3/projects/613022bf2be14f1aa7f842a6eb63cb35/users/ad292b1a14684657be07ac57f8ae0ad5/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:45.097297 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce40b5d-62be-4cda-865c-8c2bea0ff1d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=613022bf2be14f1aa7f842a6eb63cb35, user_id=ad292b1a14684657be07ac57f8ae0ad5, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:45.100611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce40b5d-62be-4cda-865c-8c2bea0ff1d7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:45.118504 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1156/2313] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:16:45 2022] PUT /identity/v3/projects/613022bf2be14f1aa7f842a6eb63cb35/users/ad292b1a14684657be07ac57f8ae0ad5/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:16:45.128296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c66c4-79ed-4953-97d0-a4cd26872c02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:45.129775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c66c4-79ed-4953-97d0-a4cd26872c02 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:45.155349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1158/2314] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:51.215532 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa047fef-b5b1-4844-99ba-40094433e60c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:51.216090 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa047fef-b5b1-4844-99ba-40094433e60c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:51.218619 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa047fef-b5b1-4844-99ba-40094433e60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:51.267952 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-aa047fef-b5b1-4844-99ba-40094433e60c None None] MFA Rules not processed for user `ad292b1a14684657be07ac57f8ae0ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:51.269368 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-aa047fef-b5b1-4844-99ba-40094433e60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:51.396130 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1157/2315] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:16:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:16:51.420210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c66c4-79ed-4953-97d0-a4cd26872c02 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:51.422794 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-803c66c4-79ed-4953-97d0-a4cd26872c02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:51.507618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:16:51.514236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c66c4-79ed-4953-97d0-a4cd26872c02 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:51.580889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6385e7-7398-4aac-92bd-747f388284d8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50ef7db27744c7ebd2ec95cecc9d1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:51.581665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6385e7-7398-4aac-92bd-747f388284d8 service glance] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:51.581942 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6385e7-7398-4aac-92bd-747f388284d8 service glance] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:51.582210 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6385e7-7398-4aac-92bd-747f388284d8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:51.678375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6385e7-7398-4aac-92bd-747f388284d8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:51.680268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6385e7-7398-4aac-92bd-747f388284d8 service glance] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:51.696814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1159/2316] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:16:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:16:52.011785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673132a5-8a47-4772-8429-3fa26a39954b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:52.011785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673132a5-8a47-4772-8429-3fa26a39954b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:52.013412 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673132a5-8a47-4772-8429-3fa26a39954b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:52.066837 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-673132a5-8a47-4772-8429-3fa26a39954b None None] MFA Rules not processed for user `ba58a4914ddf418db9e8b9dce5784cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:16:52.068576 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-673132a5-8a47-4772-8429-3fa26a39954b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:52.191830 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1158/2317] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 6 16:16:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:16:52.240691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6385e7-7398-4aac-92bd-747f388284d8 service glance] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:16:52.242179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cd6385e7-7398-4aac-92bd-747f388284d8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:16:52.347713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6385e7-7398-4aac-92bd-747f388284d8 service glance] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:16:52.355077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7740c7-f1f8-4af6-8eb7-ef4b37958522 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ece176d412574790a13d03dcca08f91a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:16:52.355692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7740c7-f1f8-4af6-8eb7-ef4b37958522 service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:16:52.355974 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7740c7-f1f8-4af6-8eb7-ef4b37958522 service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:16:52.356248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7740c7-f1f8-4af6-8eb7-ef4b37958522 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:16:52.366575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7740c7-f1f8-4af6-8eb7-ef4b37958522 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:16:52.367887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7740c7-f1f8-4af6-8eb7-ef4b37958522 service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:16:52.383396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1160/2318] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 6 16:16:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:17:00.475423 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673132a5-8a47-4772-8429-3fa26a39954b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:00.477599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-673132a5-8a47-4772-8429-3fa26a39954b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:00.489476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:00.619353 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673132a5-8a47-4772-8429-3fa26a39954b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:00.626082 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5deedcc-bbc3-4966-bf11-cc5de30cdabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:00.627160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5deedcc-bbc3-4966-bf11-cc5de30cdabf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:00.627604 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5deedcc-bbc3-4966-bf11-cc5de30cdabf admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:00.628051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5deedcc-bbc3-4966-bf11-cc5de30cdabf admin admin] PATH_INFO: `/v3/users/8354f49315884f9a888dd6231ff1a44a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:00.629828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5deedcc-bbc3-4966-bf11-cc5de30cdabf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8354f49315884f9a888dd6231ff1a44a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:00.632854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5deedcc-bbc3-4966-bf11-cc5de30cdabf admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:00.708111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a5deedcc-bbc3-4966-bf11-cc5de30cdabf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8354f49315884f9a888dd6231ff1a44a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:00.718922 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a5deedcc-bbc3-4966-bf11-cc5de30cdabf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8354f49315884f9a888dd6231ff1a44a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:00.796203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a5deedcc-bbc3-4966-bf11-cc5de30cdabf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8354f49315884f9a888dd6231ff1a44a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:00.797782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a5deedcc-bbc3-4966-bf11-cc5de30cdabf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8354f49315884f9a888dd6231ff1a44a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:00.799471 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a5deedcc-bbc3-4966-bf11-cc5de30cdabf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8354f49315884f9a888dd6231ff1a44a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:00.829924 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1159/2319] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:00 2022] DELETE /identity/v3/users/8354f49315884f9a888dd6231ff1a44a => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:00.834304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32dd74-307d-42a6-b722-7d88a8fe387c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:00.834939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32dd74-307d-42a6-b722-7d88a8fe387c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:00.835148 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32dd74-307d-42a6-b722-7d88a8fe387c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:00.861959 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-fd32dd74-307d-42a6-b722-7d88a8fe387c None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:00.863248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fd32dd74-307d-42a6-b722-7d88a8fe387c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:00.968683 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:01.005360 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1161/2320] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:01.020122 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5deedcc-bbc3-4966-bf11-cc5de30cdabf admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:01.021705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a5deedcc-bbc3-4966-bf11-cc5de30cdabf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:01.170344 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5deedcc-bbc3-4966-bf11-cc5de30cdabf admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:01.174770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19b008-f9fb-481c-886c-826bea900aa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:01.175591 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19b008-f9fb-481c-886c-826bea900aa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:01.176049 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19b008-f9fb-481c-886c-826bea900aa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:01.176328 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19b008-f9fb-481c-886c-826bea900aa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:01.190931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19b008-f9fb-481c-886c-826bea900aa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:01.192500 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19b008-f9fb-481c-886c-826bea900aa2 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:01.215179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1160/2321] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:17:01.700530 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32dd74-307d-42a6-b722-7d88a8fe387c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:01.702164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fd32dd74-307d-42a6-b722-7d88a8fe387c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:01.726467 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32dd74-307d-42a6-b722-7d88a8fe387c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:01.737116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a9248-5509-4442-989b-6ab25a8a7053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:01.737775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a9248-5509-4442-989b-6ab25a8a7053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:01.738202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a9248-5509-4442-989b-6ab25a8a7053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:01.738437 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a9248-5509-4442-989b-6ab25a8a7053 admin admin] PATH_INFO: `/v3/projects/ea6f812fb0e74be3a827cc8d47d54ad2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:01.739564 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a9248-5509-4442-989b-6ab25a8a7053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea6f812fb0e74be3a827cc8d47d54ad2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:01.741754 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a9248-5509-4442-989b-6ab25a8a7053 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:01.805576 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-764a9248-5509-4442-989b-6ab25a8a7053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea6f812fb0e74be3a827cc8d47d54ad2', 'request_id': 'req-764a9248-5509-4442-989b-6ab25a8a7053'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:01.812676 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-764a9248-5509-4442-989b-6ab25a8a7053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea6f812fb0e74be3a827cc8d47d54ad2', 'request_id': 'req-764a9248-5509-4442-989b-6ab25a8a7053'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:01.820256 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-764a9248-5509-4442-989b-6ab25a8a7053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea6f812fb0e74be3a827cc8d47d54ad2', 'request_id': 'req-764a9248-5509-4442-989b-6ab25a8a7053'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:01.838482 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-764a9248-5509-4442-989b-6ab25a8a7053 admin admin] The token cache is being invalidate because project ea6f812fb0e74be3a827cc8d47d54ad2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:01.838482 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-764a9248-5509-4442-989b-6ab25a8a7053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:01.839326 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-764a9248-5509-4442-989b-6ab25a8a7053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:01.843552 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1162/2322] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:01 2022] DELETE /identity/v3/projects/ea6f812fb0e74be3a827cc8d47d54ad2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:01.847273 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e948a-ed50-4e57-b1f8-4d792415505b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:01.848372 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e948a-ed50-4e57-b1f8-4d792415505b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:01.850067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e948a-ed50-4e57-b1f8-4d792415505b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:01.879746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d32e948a-ed50-4e57-b1f8-4d792415505b None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:01.882509 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d32e948a-ed50-4e57-b1f8-4d792415505b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:01.919436 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:01.976984 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1161/2323] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:01.984835 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ef3d5-ddd4-4b70-b82f-eef9ba9f432d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:01.985286 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ef3d5-ddd4-4b70-b82f-eef9ba9f432d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:01.985622 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ef3d5-ddd4-4b70-b82f-eef9ba9f432d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:02.010487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-623ef3d5-ddd4-4b70-b82f-eef9ba9f432d None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:02.011733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-623ef3d5-ddd4-4b70-b82f-eef9ba9f432d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:02.025690 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1163/2324] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:17:02.026428 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e948a-ed50-4e57-b1f8-4d792415505b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:02.038341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e948a-ed50-4e57-b1f8-4d792415505b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:02.047345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c4c1a-44f8-41bc-9ba1-1f9cfa1de22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:02.048001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c4c1a-44f8-41bc-9ba1-1f9cfa1de22c admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:02.048363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c4c1a-44f8-41bc-9ba1-1f9cfa1de22c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:02.048675 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c4c1a-44f8-41bc-9ba1-1f9cfa1de22c admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:02.049420 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c4c1a-44f8-41bc-9ba1-1f9cfa1de22c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:02.049736 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c4c1a-44f8-41bc-9ba1-1f9cfa1de22c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:02.051409 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c4c1a-44f8-41bc-9ba1-1f9cfa1de22c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:02.069502 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1162/2325] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:17:02.075507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ef3d5-ddd4-4b70-b82f-eef9ba9f432d None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:02.089366 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ef3d5-ddd4-4b70-b82f-eef9ba9f432d None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:02.096625 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61eeda1-65ec-46df-a920-cd6d67db1000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:02.097288 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61eeda1-65ec-46df-a920-cd6d67db1000 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:02.097662 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61eeda1-65ec-46df-a920-cd6d67db1000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:02.097938 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61eeda1-65ec-46df-a920-cd6d67db1000 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:02.098592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61eeda1-65ec-46df-a920-cd6d67db1000 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:02.100640 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61eeda1-65ec-46df-a920-cd6d67db1000 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:02.142689 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1164/2326] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:02 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:17:02.144214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c4c1a-44f8-41bc-9ba1-1f9cfa1de22c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:02.161817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c4c1a-44f8-41bc-9ba1-1f9cfa1de22c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:02.263414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c561359-a4e6-4156-9006-6f9751e4cd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:02.263953 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c561359-a4e6-4156-9006-6f9751e4cd23 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:02.264231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c561359-a4e6-4156-9006-6f9751e4cd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:02.264523 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c561359-a4e6-4156-9006-6f9751e4cd23 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:02.265155 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c561359-a4e6-4156-9006-6f9751e4cd23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:02.267783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c561359-a4e6-4156-9006-6f9751e4cd23 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:02.306314 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1163/2327] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:02 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:02.310054 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61eeda1-65ec-46df-a920-cd6d67db1000 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:02.329578 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61eeda1-65ec-46df-a920-cd6d67db1000 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:02.337732 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cbdf9-e5ea-495c-a843-8dd916d4fd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:02.339096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cbdf9-e5ea-495c-a843-8dd916d4fd5c admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:02.339555 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cbdf9-e5ea-495c-a843-8dd916d4fd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:02.339933 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cbdf9-e5ea-495c-a843-8dd916d4fd5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:02.340721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9cbdf9-e5ea-495c-a843-8dd916d4fd5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:02.344810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9cbdf9-e5ea-495c-a843-8dd916d4fd5c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:02.366631 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1165/2328] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:02.371895 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c561359-a4e6-4156-9006-6f9751e4cd23 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:02.410113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c561359-a4e6-4156-9006-6f9751e4cd23 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:02.455123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc17517-28d0-4e3f-8275-66b11960a916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:02.456053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc17517-28d0-4e3f-8275-66b11960a916 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:02.456705 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc17517-28d0-4e3f-8275-66b11960a916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:02.457124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc17517-28d0-4e3f-8275-66b11960a916 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:02.458118 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc17517-28d0-4e3f-8275-66b11960a916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:02.460988 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc17517-28d0-4e3f-8275-66b11960a916 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:02.477772 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1164/2329] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:02.482872 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cbdf9-e5ea-495c-a843-8dd916d4fd5c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:02.505351 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cbdf9-e5ea-495c-a843-8dd916d4fd5c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:02.515571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde1898-817a-4045-9fdf-e99781180097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:02.516750 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde1898-817a-4045-9fdf-e99781180097 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:02.517036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde1898-817a-4045-9fdf-e99781180097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:02.517314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde1898-817a-4045-9fdf-e99781180097 admin admin] PATH_INFO: `/v3/projects/86aeab3cef434e318c8f051cf28665d9/users/102453fccc67479d9b02c5d87a639466/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:02.554181 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde1898-817a-4045-9fdf-e99781180097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86aeab3cef434e318c8f051cf28665d9, user_id=102453fccc67479d9b02c5d87a639466, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:02.584724 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde1898-817a-4045-9fdf-e99781180097 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:02.647470 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1166/2330] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:02 2022] PUT /identity/v3/projects/86aeab3cef434e318c8f051cf28665d9/users/102453fccc67479d9b02c5d87a639466/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:02.654260 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6f90e-065a-43eb-8655-d3b0bd1ba64e None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:02.654713 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6f90e-065a-43eb-8655-d3b0bd1ba64e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:02.655049 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6f90e-065a-43eb-8655-d3b0bd1ba64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:02.750094 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-3cb6f90e-065a-43eb-8655-d3b0bd1ba64e None None] MFA Rules not processed for user `102453fccc67479d9b02c5d87a639466`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:02.753676 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3cb6f90e-065a-43eb-8655-d3b0bd1ba64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:02.973504 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1165/2331] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:03.023443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde1898-817a-4045-9fdf-e99781180097 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:03.025300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2bde1898-817a-4045-9fdf-e99781180097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:03.164014 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde1898-817a-4045-9fdf-e99781180097 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:03.174769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09427ecc-a090-4b0e-89f3-ee42dee82ce4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:03.179112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09427ecc-a090-4b0e-89f3-ee42dee82ce4 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:03.179697 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09427ecc-a090-4b0e-89f3-ee42dee82ce4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:03.181117 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09427ecc-a090-4b0e-89f3-ee42dee82ce4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:03.213367 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09427ecc-a090-4b0e-89f3-ee42dee82ce4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:03.216186 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09427ecc-a090-4b0e-89f3-ee42dee82ce4 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:03.244117 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1167/2332] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:17:03.309703 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6f90e-065a-43eb-8655-d3b0bd1ba64e None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:03.348138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:03.402317 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6f90e-065a-43eb-8655-d3b0bd1ba64e None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:03.407753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c01b8-9cce-4cef-acf3-80c64d3de743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:03.409038 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c01b8-9cce-4cef-acf3-80c64d3de743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:03.409038 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c01b8-9cce-4cef-acf3-80c64d3de743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:03.409038 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c01b8-9cce-4cef-acf3-80c64d3de743 admin admin] PATH_INFO: `/v3/users/102453fccc67479d9b02c5d87a639466` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:03.410441 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c01b8-9cce-4cef-acf3-80c64d3de743 admin admin] RBAC: Authorizing `identity:delete_user(user_id=102453fccc67479d9b02c5d87a639466)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:03.412443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c01b8-9cce-4cef-acf3-80c64d3de743 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:03.531224 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-195c01b8-9cce-4cef-acf3-80c64d3de743 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '102453fccc67479d9b02c5d87a639466'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:03.567310 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-195c01b8-9cce-4cef-acf3-80c64d3de743 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '102453fccc67479d9b02c5d87a639466'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:03.606124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-195c01b8-9cce-4cef-acf3-80c64d3de743 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '102453fccc67479d9b02c5d87a639466'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:03.606951 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-195c01b8-9cce-4cef-acf3-80c64d3de743 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '102453fccc67479d9b02c5d87a639466'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:03.608127 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-195c01b8-9cce-4cef-acf3-80c64d3de743 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '102453fccc67479d9b02c5d87a639466'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:03.634373 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1166/2333] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:03 2022] DELETE /identity/v3/users/102453fccc67479d9b02c5d87a639466 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:03.639787 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66a651-6b7d-4c43-b3f2-604cbf07f768 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:03.640474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66a651-6b7d-4c43-b3f2-604cbf07f768 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:03.640976 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66a651-6b7d-4c43-b3f2-604cbf07f768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:03.715276 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ef66a651-6b7d-4c43-b3f2-604cbf07f768 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:03.716572 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ef66a651-6b7d-4c43-b3f2-604cbf07f768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:03.723559 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:03.875844 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1168/2334] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:17:03.887118 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c01b8-9cce-4cef-acf3-80c64d3de743 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:03.888849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-195c01b8-9cce-4cef-acf3-80c64d3de743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:03.990195 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c01b8-9cce-4cef-acf3-80c64d3de743 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:04.000027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7cf02a-c712-4f1a-a0a7-0b593bd056ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:04.000027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7cf02a-c712-4f1a-a0a7-0b593bd056ca service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.000027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7cf02a-c712-4f1a-a0a7-0b593bd056ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.000027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7cf02a-c712-4f1a-a0a7-0b593bd056ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.010578 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7cf02a-c712-4f1a-a0a7-0b593bd056ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:04.012764 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7cf02a-c712-4f1a-a0a7-0b593bd056ca service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:04.025204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1167/2335] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:04.218757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4384c-4e93-45c7-afdc-48871e8b9824 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.219277 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4384c-4e93-45c7-afdc-48871e8b9824 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.219470 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4384c-4e93-45c7-afdc-48871e8b9824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.252043 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-37d4384c-4e93-45c7-afdc-48871e8b9824 None None] MFA Rules not processed for user `ac26eb671f1a4a889e8d9c7c35a70019`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:04.253539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-37d4384c-4e93-45c7-afdc-48871e8b9824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:04.323850 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1169/2336] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:04.359883 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7cf02a-c712-4f1a-a0a7-0b593bd056ca service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:04.361407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4d7cf02a-c712-4f1a-a0a7-0b593bd056ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:04.381167 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7cf02a-c712-4f1a-a0a7-0b593bd056ca service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:04.389524 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abfd53-385f-457a-8814-eaf865c53c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:04.390038 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abfd53-385f-457a-8814-eaf865c53c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.390466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abfd53-385f-457a-8814-eaf865c53c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.391450 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abfd53-385f-457a-8814-eaf865c53c36 admin admin] PATH_INFO: `/v3/projects/86aeab3cef434e318c8f051cf28665d9` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.393090 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14abfd53-385f-457a-8814-eaf865c53c36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86aeab3cef434e318c8f051cf28665d9)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:04.395278 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14abfd53-385f-457a-8814-eaf865c53c36 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:04.454245 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:04.458449 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-14abfd53-385f-457a-8814-eaf865c53c36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86aeab3cef434e318c8f051cf28665d9', 'request_id': 'req-14abfd53-385f-457a-8814-eaf865c53c36'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:04.473331 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-14abfd53-385f-457a-8814-eaf865c53c36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86aeab3cef434e318c8f051cf28665d9', 'request_id': 'req-14abfd53-385f-457a-8814-eaf865c53c36'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:04.480961 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-14abfd53-385f-457a-8814-eaf865c53c36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86aeab3cef434e318c8f051cf28665d9', 'request_id': 'req-14abfd53-385f-457a-8814-eaf865c53c36'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:04.493197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-14abfd53-385f-457a-8814-eaf865c53c36 admin admin] The token cache is being invalidate because project 86aeab3cef434e318c8f051cf28665d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:04.493481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-14abfd53-385f-457a-8814-eaf865c53c36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:04.494307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-14abfd53-385f-457a-8814-eaf865c53c36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:04.498826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1168/2337] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:04 2022] DELETE /identity/v3/projects/86aeab3cef434e318c8f051cf28665d9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:04.501190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32aa619-8458-4981-bf54-67f26a5b7b36 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.501476 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32aa619-8458-4981-bf54-67f26a5b7b36 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.501877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32aa619-8458-4981-bf54-67f26a5b7b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.522468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b32aa619-8458-4981-bf54-67f26a5b7b36 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:04.523743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b32aa619-8458-4981-bf54-67f26a5b7b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:04.596279 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1170/2338] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:04.605522 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896e539-2c21-4e9e-8c9c-a726e451f50b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.605895 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896e539-2c21-4e9e-8c9c-a726e451f50b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.606574 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896e539-2c21-4e9e-8c9c-a726e451f50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.637859 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e896e539-2c21-4e9e-8c9c-a726e451f50b None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:04.639361 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e896e539-2c21-4e9e-8c9c-a726e451f50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:04.649635 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1169/2339] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:04.654025 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32aa619-8458-4981-bf54-67f26a5b7b36 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:04.665413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32aa619-8458-4981-bf54-67f26a5b7b36 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:04.671831 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207cef3-1663-4f4b-a70b-b0edc651711e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:04.672485 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207cef3-1663-4f4b-a70b-b0edc651711e admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.672763 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207cef3-1663-4f4b-a70b-b0edc651711e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.673046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207cef3-1663-4f4b-a70b-b0edc651711e admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.673700 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207cef3-1663-4f4b-a70b-b0edc651711e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:04.674002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207cef3-1663-4f4b-a70b-b0edc651711e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:04.675662 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207cef3-1663-4f4b-a70b-b0edc651711e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:04.688724 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1171/2340] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:17:04.693734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896e539-2c21-4e9e-8c9c-a726e451f50b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:04.704039 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896e539-2c21-4e9e-8c9c-a726e451f50b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:04.709010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd220ae-fcc4-4379-8e03-a53500c59656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:04.709616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd220ae-fcc4-4379-8e03-a53500c59656 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.710436 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd220ae-fcc4-4379-8e03-a53500c59656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.710759 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd220ae-fcc4-4379-8e03-a53500c59656 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.711412 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd220ae-fcc4-4379-8e03-a53500c59656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:04.713384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd220ae-fcc4-4379-8e03-a53500c59656 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:04.741496 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1170/2341] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:04 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:04.745978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207cef3-1663-4f4b-a70b-b0edc651711e admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:04.756819 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207cef3-1663-4f4b-a70b-b0edc651711e admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:04.801077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:04.819077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ec5fc-988f-4192-98e8-29f7350813f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:04.819247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ec5fc-988f-4192-98e8-29f7350813f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.819529 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ec5fc-988f-4192-98e8-29f7350813f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.819800 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ec5fc-988f-4192-98e8-29f7350813f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.821910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ec5fc-988f-4192-98e8-29f7350813f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:04.822640 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ec5fc-988f-4192-98e8-29f7350813f2 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:04.854745 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1172/2342] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:04 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:17:04.855314 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd220ae-fcc4-4379-8e03-a53500c59656 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:04.866059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd220ae-fcc4-4379-8e03-a53500c59656 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:04.870752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770ca76-a230-4664-8e2b-d00433f958b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:04.871359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770ca76-a230-4664-8e2b-d00433f958b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.871634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770ca76-a230-4664-8e2b-d00433f958b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.871927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770ca76-a230-4664-8e2b-d00433f958b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.872537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770ca76-a230-4664-8e2b-d00433f958b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:04.874116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770ca76-a230-4664-8e2b-d00433f958b0 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:04.887828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1171/2343] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:17:04.894626 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ec5fc-988f-4192-98e8-29f7350813f2 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:04.905081 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ec5fc-988f-4192-98e8-29f7350813f2 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:04.912153 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297496ca-b82a-4d09-9029-0ed3ea559d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:04.912805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297496ca-b82a-4d09-9029-0ed3ea559d08 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.913080 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297496ca-b82a-4d09-9029-0ed3ea559d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.913347 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297496ca-b82a-4d09-9029-0ed3ea559d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.914086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297496ca-b82a-4d09-9029-0ed3ea559d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:04.915820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297496ca-b82a-4d09-9029-0ed3ea559d08 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:04.929266 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1173/2344] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:04.933467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770ca76-a230-4664-8e2b-d00433f958b0 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:04.946171 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770ca76-a230-4664-8e2b-d00433f958b0 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:04.951722 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e03ad-bb33-42f3-8049-2c7b653b2ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:04.952570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e03ad-bb33-42f3-8049-2c7b653b2ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:04.952850 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e03ad-bb33-42f3-8049-2c7b653b2ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:04.953699 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e03ad-bb33-42f3-8049-2c7b653b2ffe admin admin] PATH_INFO: `/v3/projects/17e81ff9ac5c4468989eb31ba2130e0d/users/b266573c02ab476da327907dcfed5583/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:04.977316 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e03ad-bb33-42f3-8049-2c7b653b2ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=17e81ff9ac5c4468989eb31ba2130e0d, user_id=b266573c02ab476da327907dcfed5583, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:04.981120 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e03ad-bb33-42f3-8049-2c7b653b2ffe admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:05.010001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793f5ef-d3f0-4529-8f11-51d214383a7c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:05.010633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793f5ef-d3f0-4529-8f11-51d214383a7c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:05.011130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793f5ef-d3f0-4529-8f11-51d214383a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:05.015648 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1172/2345] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:04 2022] PUT /identity/v3/projects/17e81ff9ac5c4468989eb31ba2130e0d/users/b266573c02ab476da327907dcfed5583/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:17:05.038603 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-1793f5ef-d3f0-4529-8f11-51d214383a7c None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:05.040115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1793f5ef-d3f0-4529-8f11-51d214383a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:05.116623 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1174/2346] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:05.127459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e03ad-bb33-42f3-8049-2c7b653b2ffe admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:05.129226 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-429e03ad-bb33-42f3-8049-2c7b653b2ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:05.205227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e03ad-bb33-42f3-8049-2c7b653b2ffe admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:05.210115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f5f04-cd67-4f90-8fe6-3ef2d032afbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:05.210553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f5f04-cd67-4f90-8fe6-3ef2d032afbe service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:05.210926 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f5f04-cd67-4f90-8fe6-3ef2d032afbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:05.211209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f5f04-cd67-4f90-8fe6-3ef2d032afbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:05.223718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f5f04-cd67-4f90-8fe6-3ef2d032afbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:05.225259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f5f04-cd67-4f90-8fe6-3ef2d032afbe service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:05.236841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1173/2347] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:06.094619 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca3079-bcd2-49fa-b158-82aae2fa7617 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:06.095613 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca3079-bcd2-49fa-b158-82aae2fa7617 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:06.096075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca3079-bcd2-49fa-b158-82aae2fa7617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:06.131977 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4fca3079-bcd2-49fa-b158-82aae2fa7617 None None] MFA Rules not processed for user `b266573c02ab476da327907dcfed5583`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:06.132329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4fca3079-bcd2-49fa-b158-82aae2fa7617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:06.218172 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1175/2348] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:17:06.229901 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f5f04-cd67-4f90-8fe6-3ef2d032afbe service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:06.231363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cb0f5f04-cd67-4f90-8fe6-3ef2d032afbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:06.315048 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f5f04-cd67-4f90-8fe6-3ef2d032afbe service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:06.318860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddacc21-8a32-454c-9bd3-6361a321498f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:06.319490 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddacc21-8a32-454c-9bd3-6361a321498f service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:06.319768 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddacc21-8a32-454c-9bd3-6361a321498f service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:06.320040 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddacc21-8a32-454c-9bd3-6361a321498f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:06.330749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddacc21-8a32-454c-9bd3-6361a321498f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:06.332204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddacc21-8a32-454c-9bd3-6361a321498f service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:06.342363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1174/2349] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:06 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:17:11.961365 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290af35f-dff8-4f6d-92fa-bad06b869bdb None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:11.961958 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290af35f-dff8-4f6d-92fa-bad06b869bdb None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:11.961958 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290af35f-dff8-4f6d-92fa-bad06b869bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:11.986238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-290af35f-dff8-4f6d-92fa-bad06b869bdb None None] MFA Rules not processed for user `b266573c02ab476da327907dcfed5583`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:11.987998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-290af35f-dff8-4f6d-92fa-bad06b869bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:11.996003 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1176/2350] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:15.753547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b49e9-4d9c-4b45-9858-ec1a50464323 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:15.754050 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b49e9-4d9c-4b45-9858-ec1a50464323 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:15.754149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b49e9-4d9c-4b45-9858-ec1a50464323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:15.777614 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-601b49e9-4d9c-4b45-9858-ec1a50464323 None None] MFA Rules not processed for user `ba58a4914ddf418db9e8b9dce5784cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:15.778875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-601b49e9-4d9c-4b45-9858-ec1a50464323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:15.791154 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:15.915258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1175/2351] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 6 16:17:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:15.937173 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290af35f-dff8-4f6d-92fa-bad06b869bdb None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:15.939417 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-290af35f-dff8-4f6d-92fa-bad06b869bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:15.979527 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b49e9-4d9c-4b45-9858-ec1a50464323 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:15.981476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-601b49e9-4d9c-4b45-9858-ec1a50464323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:16.009573 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b49e9-4d9c-4b45-9858-ec1a50464323 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:16.032570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a3abf-e71f-4567-b8f4-a65173cde831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:16.034517 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a3abf-e71f-4567-b8f4-a65173cde831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:16.035755 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a3abf-e71f-4567-b8f4-a65173cde831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:16.039017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a3abf-e71f-4567-b8f4-a65173cde831 admin admin] PATH_INFO: `/v3/users/6ad4f18630864aa399bc274240078a75` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:16.040773 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a3abf-e71f-4567-b8f4-a65173cde831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ad4f18630864aa399bc274240078a75)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:16.048413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a3abf-e71f-4567-b8f4-a65173cde831 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:16.072634 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:16.121765 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290af35f-dff8-4f6d-92fa-bad06b869bdb None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:16.133844 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae6aa6-7615-48f2-a3b1-ab049c0d79fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ece176d412574790a13d03dcca08f91a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:16.134762 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae6aa6-7615-48f2-a3b1-ab049c0d79fb service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:16.135236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae6aa6-7615-48f2-a3b1-ab049c0d79fb service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:16.135862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae6aa6-7615-48f2-a3b1-ab049c0d79fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:16.153841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae6aa6-7615-48f2-a3b1-ab049c0d79fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:16.156397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae6aa6-7615-48f2-a3b1-ab049c0d79fb service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:16.183412 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1177/2352] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 6 16:17:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:17:16.202930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7f8a3abf-e71f-4567-b8f4-a65173cde831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ad4f18630864aa399bc274240078a75'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:16.222776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7f8a3abf-e71f-4567-b8f4-a65173cde831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ad4f18630864aa399bc274240078a75'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:16.242710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7f8a3abf-e71f-4567-b8f4-a65173cde831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ad4f18630864aa399bc274240078a75'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:16.243785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7f8a3abf-e71f-4567-b8f4-a65173cde831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ad4f18630864aa399bc274240078a75'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:16.244644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7f8a3abf-e71f-4567-b8f4-a65173cde831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ad4f18630864aa399bc274240078a75'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:16.264863 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1176/2353] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:15 2022] DELETE /identity/v3/users/6ad4f18630864aa399bc274240078a75 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:16.269654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb808eb-3fc5-4573-b6f9-9093326b7aaa None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:16.269942 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb808eb-3fc5-4573-b6f9-9093326b7aaa None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:16.270212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb808eb-3fc5-4573-b6f9-9093326b7aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:16.293128 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7bb808eb-3fc5-4573-b6f9-9093326b7aaa None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:16.296412 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7bb808eb-3fc5-4573-b6f9-9093326b7aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:16.437687 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1178/2354] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:16.461760 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a3abf-e71f-4567-b8f4-a65173cde831 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:16.463673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7f8a3abf-e71f-4567-b8f4-a65173cde831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:16.601054 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a3abf-e71f-4567-b8f4-a65173cde831 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:16.612682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71b179-e570-4886-a8f2-fe99c3bd77f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:16.613328 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71b179-e570-4886-a8f2-fe99c3bd77f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:16.613606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71b179-e570-4886-a8f2-fe99c3bd77f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:16.613875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71b179-e570-4886-a8f2-fe99c3bd77f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:16.636680 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d71b179-e570-4886-a8f2-fe99c3bd77f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:16.639042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d71b179-e570-4886-a8f2-fe99c3bd77f4 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:16.674107 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1177/2355] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:17.272848 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb808eb-3fc5-4573-b6f9-9093326b7aaa None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:17.275067 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7bb808eb-3fc5-4573-b6f9-9093326b7aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:17.288509 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb808eb-3fc5-4573-b6f9-9093326b7aaa None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:17.294995 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e41025-64dd-40d8-8a93-fef0d914eb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:17.296123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e41025-64dd-40d8-8a93-fef0d914eb68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:17.296618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e41025-64dd-40d8-8a93-fef0d914eb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:17.297014 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e41025-64dd-40d8-8a93-fef0d914eb68 admin admin] PATH_INFO: `/v3/projects/5f88c60a4a8745399943e21c85d92a1d` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:17.299219 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e41025-64dd-40d8-8a93-fef0d914eb68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f88c60a4a8745399943e21c85d92a1d)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:17.301376 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e41025-64dd-40d8-8a93-fef0d914eb68 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:17.376332 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-62e41025-64dd-40d8-8a93-fef0d914eb68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f88c60a4a8745399943e21c85d92a1d', 'request_id': 'req-62e41025-64dd-40d8-8a93-fef0d914eb68'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:17.383780 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-62e41025-64dd-40d8-8a93-fef0d914eb68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f88c60a4a8745399943e21c85d92a1d', 'request_id': 'req-62e41025-64dd-40d8-8a93-fef0d914eb68'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:17.394038 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-62e41025-64dd-40d8-8a93-fef0d914eb68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f88c60a4a8745399943e21c85d92a1d', 'request_id': 'req-62e41025-64dd-40d8-8a93-fef0d914eb68'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:17.407169 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-62e41025-64dd-40d8-8a93-fef0d914eb68 admin admin] The token cache is being invalidate because project 5f88c60a4a8745399943e21c85d92a1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:17.407457 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-62e41025-64dd-40d8-8a93-fef0d914eb68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:17.408102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-62e41025-64dd-40d8-8a93-fef0d914eb68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:17.409595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1179/2356] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:17 2022] DELETE /identity/v3/projects/5f88c60a4a8745399943e21c85d92a1d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:17.414589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26635e7-9baf-43a9-8b4c-a1ad69dfbb53 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:17.415222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26635e7-9baf-43a9-8b4c-a1ad69dfbb53 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:17.415222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26635e7-9baf-43a9-8b4c-a1ad69dfbb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:17.437573 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d26635e7-9baf-43a9-8b4c-a1ad69dfbb53 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:17.439144 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d26635e7-9baf-43a9-8b4c-a1ad69dfbb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:17.478777 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:17.518309 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1178/2357] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:17.539084 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e5a9b-a094-4498-a9cb-3c3d7b1a1002 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:17.539518 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e5a9b-a094-4498-a9cb-3c3d7b1a1002 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:17.539803 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e5a9b-a094-4498-a9cb-3c3d7b1a1002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:17.561463 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-403e5a9b-a094-4498-a9cb-3c3d7b1a1002 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:17.563085 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-403e5a9b-a094-4498-a9cb-3c3d7b1a1002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:17.580687 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1180/2358] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:17.582216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26635e7-9baf-43a9-8b4c-a1ad69dfbb53 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:17.592925 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26635e7-9baf-43a9-8b4c-a1ad69dfbb53 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:17.601305 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35e275-a48b-4bac-b8ca-b1f7b32d8170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:17.601934 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35e275-a48b-4bac-b8ca-b1f7b32d8170 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:17.602207 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35e275-a48b-4bac-b8ca-b1f7b32d8170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:17.602475 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35e275-a48b-4bac-b8ca-b1f7b32d8170 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:17.603169 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35e275-a48b-4bac-b8ca-b1f7b32d8170 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:17.603459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35e275-a48b-4bac-b8ca-b1f7b32d8170 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:17.605209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35e275-a48b-4bac-b8ca-b1f7b32d8170 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:17.622765 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1179/2359] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:17:17.625809 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e5a9b-a094-4498-a9cb-3c3d7b1a1002 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:17.640495 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e5a9b-a094-4498-a9cb-3c3d7b1a1002 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:17.647727 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997abdfc-65d1-41f4-a22c-e0b037461aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:17.648739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997abdfc-65d1-41f4-a22c-e0b037461aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:17.649243 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997abdfc-65d1-41f4-a22c-e0b037461aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:17.649771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997abdfc-65d1-41f4-a22c-e0b037461aa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:17.650994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997abdfc-65d1-41f4-a22c-e0b037461aa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:17.654833 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997abdfc-65d1-41f4-a22c-e0b037461aa5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:17.702834 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1181/2360] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:17 2022] POST /identity/v3/projects => generated 399 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:17.707760 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35e275-a48b-4bac-b8ca-b1f7b32d8170 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:17.719971 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35e275-a48b-4bac-b8ca-b1f7b32d8170 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:17.806381 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccc464-dcb4-4896-b10f-3bcd65981531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:17.807215 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccc464-dcb4-4896-b10f-3bcd65981531 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:17.807559 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccc464-dcb4-4896-b10f-3bcd65981531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:17.807917 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccc464-dcb4-4896-b10f-3bcd65981531 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:17.808575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccc464-dcb4-4896-b10f-3bcd65981531 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:17.810557 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccc464-dcb4-4896-b10f-3bcd65981531 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:17.851840 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1180/2361] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:17 2022] POST /identity/v3/users => generated 300 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:17.855746 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997abdfc-65d1-41f4-a22c-e0b037461aa5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:17.870811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997abdfc-65d1-41f4-a22c-e0b037461aa5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:17.877601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713a8b4-f2ad-402b-b9ed-3522950ccf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:17.878490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713a8b4-f2ad-402b-b9ed-3522950ccf15 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:17.878905 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713a8b4-f2ad-402b-b9ed-3522950ccf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:17.879281 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713a8b4-f2ad-402b-b9ed-3522950ccf15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:17.880050 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713a8b4-f2ad-402b-b9ed-3522950ccf15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:17.881942 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713a8b4-f2ad-402b-b9ed-3522950ccf15 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:17.898875 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1182/2362] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:17.900220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccc464-dcb4-4896-b10f-3bcd65981531 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:17.913096 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccc464-dcb4-4896-b10f-3bcd65981531 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:17.918206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bad438-d486-487f-ab5b-af0cfcb9f23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:17.918858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bad438-d486-487f-ab5b-af0cfcb9f23f admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:17.919201 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bad438-d486-487f-ab5b-af0cfcb9f23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:17.919484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bad438-d486-487f-ab5b-af0cfcb9f23f admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:17.920125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bad438-d486-487f-ab5b-af0cfcb9f23f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:17.921747 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bad438-d486-487f-ab5b-af0cfcb9f23f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:17.934359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1181/2363] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:17.939413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713a8b4-f2ad-402b-b9ed-3522950ccf15 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:17.950112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713a8b4-f2ad-402b-b9ed-3522950ccf15 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:17.956214 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc4b9e-36f8-429a-aba4-f0c0d48f4753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:17.956882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc4b9e-36f8-429a-aba4-f0c0d48f4753 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:17.957160 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc4b9e-36f8-429a-aba4-f0c0d48f4753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:17.957428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc4b9e-36f8-429a-aba4-f0c0d48f4753 admin admin] PATH_INFO: `/v3/projects/741013390cb34c42a24ecd41cf74cdf4/users/922b2bbf4c2e464b88c55a95a2860bb9/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:17.975196 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc4b9e-36f8-429a-aba4-f0c0d48f4753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=741013390cb34c42a24ecd41cf74cdf4, user_id=922b2bbf4c2e464b88c55a95a2860bb9, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:17.977912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc4b9e-36f8-429a-aba4-f0c0d48f4753 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:17.993021 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:18.011142 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1183/2364] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:17 2022] PUT /identity/v3/projects/741013390cb34c42a24ecd41cf74cdf4/users/922b2bbf4c2e464b88c55a95a2860bb9/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:18.023709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e441eb7-9733-4ebb-b3b8-85313b460005 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:18.024290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e441eb7-9733-4ebb-b3b8-85313b460005 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:18.024743 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e441eb7-9733-4ebb-b3b8-85313b460005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:18.089587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-2e441eb7-9733-4ebb-b3b8-85313b460005 None None] MFA Rules not processed for user `922b2bbf4c2e464b88c55a95a2860bb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:18.091823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2e441eb7-9733-4ebb-b3b8-85313b460005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:18.224392 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1182/2365] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:18.233557 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc4b9e-36f8-429a-aba4-f0c0d48f4753 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:18.235429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a9dc4b9e-36f8-429a-aba4-f0c0d48f4753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:18.311209 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc4b9e-36f8-429a-aba4-f0c0d48f4753 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:18.316795 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652b240-4c07-4fe6-a9b0-edd04140218f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:18.317483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652b240-4c07-4fe6-a9b0-edd04140218f service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:18.317797 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652b240-4c07-4fe6-a9b0-edd04140218f service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:18.318116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652b240-4c07-4fe6-a9b0-edd04140218f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:18.328520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652b240-4c07-4fe6-a9b0-edd04140218f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:18.329970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652b240-4c07-4fe6-a9b0-edd04140218f service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:18.339767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1184/2366] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:18 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:22.689489 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ee14e-81f8-4acc-bab9-0a2cacab8849 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:22.690295 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ee14e-81f8-4acc-bab9-0a2cacab8849 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:22.690505 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ee14e-81f8-4acc-bab9-0a2cacab8849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:22.713206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1c6ee14e-81f8-4acc-bab9-0a2cacab8849 None None] MFA Rules not processed for user `ba58a4914ddf418db9e8b9dce5784cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:22.714408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1c6ee14e-81f8-4acc-bab9-0a2cacab8849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:22.742129 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:22.790820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1183/2367] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 6 16:17:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:22.804320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652b240-4c07-4fe6-a9b0-edd04140218f service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:22.805744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2652b240-4c07-4fe6-a9b0-edd04140218f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:22.883168 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652b240-4c07-4fe6-a9b0-edd04140218f service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:22.889771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b43f9-aab0-4ffd-bce4-8afda462b3d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ece176d412574790a13d03dcca08f91a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:22.890453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b43f9-aab0-4ffd-bce4-8afda462b3d3 service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:22.890867 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b43f9-aab0-4ffd-bce4-8afda462b3d3 service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:22.891197 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b43f9-aab0-4ffd-bce4-8afda462b3d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:22.900486 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b43f9-aab0-4ffd-bce4-8afda462b3d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:22.902090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b43f9-aab0-4ffd-bce4-8afda462b3d3 service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:22.912696 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1185/2368] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 6 16:17:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:17:32.778573 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=12 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:32.908136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=60 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:35.027512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ee14e-81f8-4acc-bab9-0a2cacab8849 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:35.217885 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ee14e-81f8-4acc-bab9-0a2cacab8849 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:35.227286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba9be9-f213-4224-af88-d09b403caf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:35.228114 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9be9-f213-4224-af88-d09b403caf8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:35.228498 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9be9-f213-4224-af88-d09b403caf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:35.228872 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9be9-f213-4224-af88-d09b403caf8f admin admin] PATH_INFO: `/v3/users/922b2bbf4c2e464b88c55a95a2860bb9` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:35.230864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba9be9-f213-4224-af88-d09b403caf8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=922b2bbf4c2e464b88c55a95a2860bb9)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:35.232947 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba9be9-f213-4224-af88-d09b403caf8f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:35.407794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-14ba9be9-f213-4224-af88-d09b403caf8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '922b2bbf4c2e464b88c55a95a2860bb9'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:35.430831 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-14ba9be9-f213-4224-af88-d09b403caf8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '922b2bbf4c2e464b88c55a95a2860bb9'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:35.463592 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-14ba9be9-f213-4224-af88-d09b403caf8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '922b2bbf4c2e464b88c55a95a2860bb9'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:35.475018 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-14ba9be9-f213-4224-af88-d09b403caf8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '922b2bbf4c2e464b88c55a95a2860bb9'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:35.475901 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-14ba9be9-f213-4224-af88-d09b403caf8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '922b2bbf4c2e464b88c55a95a2860bb9'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:35.499096 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1184/2369] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:35 2022] DELETE /identity/v3/users/922b2bbf4c2e464b88c55a95a2860bb9 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:35.504322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaeab24-3bd3-4a33-95a0-e56f373b4f08 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:35.504864 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaeab24-3bd3-4a33-95a0-e56f373b4f08 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:35.504864 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaeab24-3bd3-4a33-95a0-e56f373b4f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:35.549369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-cfaeab24-3bd3-4a33-95a0-e56f373b4f08 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:35.550631 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cfaeab24-3bd3-4a33-95a0-e56f373b4f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:35.658511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1186/2370] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:17:35.670749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba9be9-f213-4224-af88-d09b403caf8f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:35.674706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-14ba9be9-f213-4224-af88-d09b403caf8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:35.794060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba9be9-f213-4224-af88-d09b403caf8f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:35.801889 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57840ee-49ef-423b-85be-c6287efb104b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:35.803087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57840ee-49ef-423b-85be-c6287efb104b service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:35.803597 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57840ee-49ef-423b-85be-c6287efb104b service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:35.804051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57840ee-49ef-423b-85be-c6287efb104b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:35.822152 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57840ee-49ef-423b-85be-c6287efb104b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:35.824413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57840ee-49ef-423b-85be-c6287efb104b service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:35.857226 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1185/2371] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:36.445092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaeab24-3bd3-4a33-95a0-e56f373b4f08 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:36.448071 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cfaeab24-3bd3-4a33-95a0-e56f373b4f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:36.756253 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaeab24-3bd3-4a33-95a0-e56f373b4f08 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:36.781868 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b806b-4205-4e22-9c7b-f248b63ef7e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034d75ab747e4c0cab040e47c6a6236e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:36.782585 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b806b-4205-4e22-9c7b-f248b63ef7e6 service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:36.785217 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b806b-4205-4e22-9c7b-f248b63ef7e6 service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:36.785429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b806b-4205-4e22-9c7b-f248b63ef7e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:36.790024 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ce5b806b-4205-4e22-9c7b-f248b63ef7e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:36.831648 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57840ee-49ef-423b-85be-c6287efb104b service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:36.836584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c57840ee-49ef-423b-85be-c6287efb104b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:36.847721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b806b-4205-4e22-9c7b-f248b63ef7e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:36.857362 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b806b-4205-4e22-9c7b-f248b63ef7e6 service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:36.921420 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1187/2372] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:36.924782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57840ee-49ef-423b-85be-c6287efb104b service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:36.972622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e735a-1142-45be-8fb6-f1ac6c725b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:36.973780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e735a-1142-45be-8fb6-f1ac6c725b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:36.974270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e735a-1142-45be-8fb6-f1ac6c725b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:36.974844 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e735a-1142-45be-8fb6-f1ac6c725b25 admin admin] PATH_INFO: `/v3/projects/741013390cb34c42a24ecd41cf74cdf4` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:36.990692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e735a-1142-45be-8fb6-f1ac6c725b25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=741013390cb34c42a24ecd41cf74cdf4)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:36.993734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e735a-1142-45be-8fb6-f1ac6c725b25 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:37.055338 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:37.163365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-054e735a-1142-45be-8fb6-f1ac6c725b25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '741013390cb34c42a24ecd41cf74cdf4', 'request_id': 'req-054e735a-1142-45be-8fb6-f1ac6c725b25'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:37.188424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-054e735a-1142-45be-8fb6-f1ac6c725b25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '741013390cb34c42a24ecd41cf74cdf4', 'request_id': 'req-054e735a-1142-45be-8fb6-f1ac6c725b25'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:37.214834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-054e735a-1142-45be-8fb6-f1ac6c725b25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '741013390cb34c42a24ecd41cf74cdf4', 'request_id': 'req-054e735a-1142-45be-8fb6-f1ac6c725b25'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:37.263450 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-054e735a-1142-45be-8fb6-f1ac6c725b25 admin admin] The token cache is being invalidate because project 741013390cb34c42a24ecd41cf74cdf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:37.266654 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-054e735a-1142-45be-8fb6-f1ac6c725b25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:37.272169 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-054e735a-1142-45be-8fb6-f1ac6c725b25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:37.280292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1186/2373] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:36 2022] DELETE /identity/v3/projects/741013390cb34c42a24ecd41cf74cdf4 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:37.285682 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b8278-375d-42a1-94be-d3b89f4ad92a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:37.286565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b8278-375d-42a1-94be-d3b89f4ad92a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:37.287212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b8278-375d-42a1-94be-d3b89f4ad92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:37.361348 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e17b8278-375d-42a1-94be-d3b89f4ad92a None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:37.363445 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e17b8278-375d-42a1-94be-d3b89f4ad92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:37.534594 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1188/2374] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:37.546424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f14dd-35d4-4929-9ac5-8f329ab2a53f None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:37.547249 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f14dd-35d4-4929-9ac5-8f329ab2a53f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:37.547356 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f14dd-35d4-4929-9ac5-8f329ab2a53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:37.626241 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-447f14dd-35d4-4929-9ac5-8f329ab2a53f None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:37.628369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-447f14dd-35d4-4929-9ac5-8f329ab2a53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:37.655620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1187/2375] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:37.662329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b8278-375d-42a1-94be-d3b89f4ad92a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:37.675735 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b8278-375d-42a1-94be-d3b89f4ad92a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:37.691110 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79f716-569d-4c78-a1a9-183fa4de27b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:37.698703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79f716-569d-4c78-a1a9-183fa4de27b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:37.699351 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79f716-569d-4c78-a1a9-183fa4de27b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:37.699863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79f716-569d-4c78-a1a9-183fa4de27b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:37.700791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79f716-569d-4c78-a1a9-183fa4de27b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:37.701327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79f716-569d-4c78-a1a9-183fa4de27b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:37.704504 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79f716-569d-4c78-a1a9-183fa4de27b9 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:37.761151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1189/2376] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:17:37.764799 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f14dd-35d4-4929-9ac5-8f329ab2a53f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:37.798842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f14dd-35d4-4929-9ac5-8f329ab2a53f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:37.813404 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667b308-90e2-456c-bcc7-22493797fbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:37.814563 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667b308-90e2-456c-bcc7-22493797fbba admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:37.814870 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667b308-90e2-456c-bcc7-22493797fbba admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:37.815507 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667b308-90e2-456c-bcc7-22493797fbba admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:37.816095 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667b308-90e2-456c-bcc7-22493797fbba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:37.820451 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667b308-90e2-456c-bcc7-22493797fbba admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:37.907499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1188/2377] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:37 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:37.912339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79f716-569d-4c78-a1a9-183fa4de27b9 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:37.938620 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79f716-569d-4c78-a1a9-183fa4de27b9 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:37.989587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:38.139711 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d85f25-34ab-4d6a-b095-a09702eff9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:38.142986 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d85f25-34ab-4d6a-b095-a09702eff9aa admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:38.143383 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d85f25-34ab-4d6a-b095-a09702eff9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:38.143646 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d85f25-34ab-4d6a-b095-a09702eff9aa admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:38.144237 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d85f25-34ab-4d6a-b095-a09702eff9aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:38.146340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d85f25-34ab-4d6a-b095-a09702eff9aa admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:38.196829 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1190/2378] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:37 2022] POST /identity/v3/users => generated 286 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:17:38.209088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667b308-90e2-456c-bcc7-22493797fbba admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:38.228001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667b308-90e2-456c-bcc7-22493797fbba admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:38.243510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd20724-50a7-49de-abb8-36e649048697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:38.244519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd20724-50a7-49de-abb8-36e649048697 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:38.246426 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd20724-50a7-49de-abb8-36e649048697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:38.249166 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd20724-50a7-49de-abb8-36e649048697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:38.253775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd20724-50a7-49de-abb8-36e649048697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:38.255810 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd20724-50a7-49de-abb8-36e649048697 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:38.314837 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1189/2379] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:38.319962 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d85f25-34ab-4d6a-b095-a09702eff9aa admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:38.362181 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d85f25-34ab-4d6a-b095-a09702eff9aa admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:38.372751 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82973ee5-21fb-42f5-8bc8-87893c7f9d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:38.373403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82973ee5-21fb-42f5-8bc8-87893c7f9d4e admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:38.373785 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82973ee5-21fb-42f5-8bc8-87893c7f9d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:38.373938 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82973ee5-21fb-42f5-8bc8-87893c7f9d4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:38.374615 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82973ee5-21fb-42f5-8bc8-87893c7f9d4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:38.380322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82973ee5-21fb-42f5-8bc8-87893c7f9d4e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:38.423694 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1191/2380] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:38.426868 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd20724-50a7-49de-abb8-36e649048697 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:38.450172 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd20724-50a7-49de-abb8-36e649048697 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:38.466630 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97156a-57b1-402a-b1c1-e91132b0b694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:38.467377 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97156a-57b1-402a-b1c1-e91132b0b694 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:38.467671 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97156a-57b1-402a-b1c1-e91132b0b694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:38.467982 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97156a-57b1-402a-b1c1-e91132b0b694 admin admin] PATH_INFO: `/v3/projects/abf281d8694e486da6d616fe43107474/users/e0eaab966616458592a89cdef9cf63c3/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:38.505030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97156a-57b1-402a-b1c1-e91132b0b694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abf281d8694e486da6d616fe43107474, user_id=e0eaab966616458592a89cdef9cf63c3, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:38.510149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97156a-57b1-402a-b1c1-e91132b0b694 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:38.560152 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1190/2381] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:38 2022] PUT /identity/v3/projects/abf281d8694e486da6d616fe43107474/users/e0eaab966616458592a89cdef9cf63c3/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:38.566092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d73a58-9f32-4f19-9e67-0fa2ddf0d2ee None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:38.566694 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d73a58-9f32-4f19-9e67-0fa2ddf0d2ee None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:38.567300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d73a58-9f32-4f19-9e67-0fa2ddf0d2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:38.605577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-71d73a58-9f32-4f19-9e67-0fa2ddf0d2ee None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:38.606961 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-71d73a58-9f32-4f19-9e67-0fa2ddf0d2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:38.803693 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1192/2382] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:38.997671 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97156a-57b1-402a-b1c1-e91132b0b694 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:39.003789 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ed97156a-57b1-402a-b1c1-e91132b0b694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:39.595186 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97156a-57b1-402a-b1c1-e91132b0b694 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:39.646012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d4561-16c6-4e4c-9263-de4302bbf78e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:39.650389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d4561-16c6-4e4c-9263-de4302bbf78e service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:39.652039 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d4561-16c6-4e4c-9263-de4302bbf78e service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:39.653969 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d4561-16c6-4e4c-9263-de4302bbf78e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:39.722383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620d4561-16c6-4e4c-9263-de4302bbf78e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:39.725351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620d4561-16c6-4e4c-9263-de4302bbf78e service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:39.782395 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1191/2383] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 785 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:43.369955 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a698f-42fe-4170-8cf1-69ebc8295303 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:43.371974 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a698f-42fe-4170-8cf1-69ebc8295303 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:43.372701 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a698f-42fe-4170-8cf1-69ebc8295303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:43.494802 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e40a698f-42fe-4170-8cf1-69ebc8295303 None None] MFA Rules not processed for user `eda6839efaab4317a0eda1f47f1656a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:43.497135 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e40a698f-42fe-4170-8cf1-69ebc8295303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:43.728632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1193/2384] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:45.290828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f5277-21f5-4be4-ad18-80e92215572d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:45.293689 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f5277-21f5-4be4-ad18-80e92215572d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:45.294364 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f5277-21f5-4be4-ad18-80e92215572d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:45.335108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-ed5f5277-21f5-4be4-ad18-80e92215572d None None] MFA Rules not processed for user `ba58a4914ddf418db9e8b9dce5784cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:45.336535 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ed5f5277-21f5-4be4-ad18-80e92215572d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:45.455205 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:45.503153 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1192/2385] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 6 16:17:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:45.522322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a698f-42fe-4170-8cf1-69ebc8295303 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:45.526218 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e40a698f-42fe-4170-8cf1-69ebc8295303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:45.652945 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:45.704753 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a698f-42fe-4170-8cf1-69ebc8295303 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:45.710702 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780994c5-f40b-4871-ab12-17ff7e0342f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ece176d412574790a13d03dcca08f91a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:45.711224 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780994c5-f40b-4871-ab12-17ff7e0342f4 service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:45.711573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780994c5-f40b-4871-ab12-17ff7e0342f4 service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:45.711909 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780994c5-f40b-4871-ab12-17ff7e0342f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:45.722156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780994c5-f40b-4871-ab12-17ff7e0342f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:45.724105 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780994c5-f40b-4871-ab12-17ff7e0342f4 service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:45.739831 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1194/2386] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 6 16:17:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:17:48.868119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f5277-21f5-4be4-ad18-80e92215572d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:48.870153 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ed5f5277-21f5-4be4-ad18-80e92215572d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:48.911973 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f5277-21f5-4be4-ad18-80e92215572d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:48.918921 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6598e-c321-4723-9739-c263ad532242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:48.919911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6598e-c321-4723-9739-c263ad532242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:48.920529 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6598e-c321-4723-9739-c263ad532242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:48.921060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6598e-c321-4723-9739-c263ad532242 admin admin] PATH_INFO: `/v3/users/b24f1542cddf4fc9aa8003952e26cb96` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:48.929242 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6598e-c321-4723-9739-c263ad532242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b24f1542cddf4fc9aa8003952e26cb96)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:48.931987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6598e-c321-4723-9739-c263ad532242 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:49.028934 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1ac6598e-c321-4723-9739-c263ad532242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b24f1542cddf4fc9aa8003952e26cb96'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:49.044033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1ac6598e-c321-4723-9739-c263ad532242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b24f1542cddf4fc9aa8003952e26cb96'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:49.072431 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1ac6598e-c321-4723-9739-c263ad532242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b24f1542cddf4fc9aa8003952e26cb96'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:49.075899 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1ac6598e-c321-4723-9739-c263ad532242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b24f1542cddf4fc9aa8003952e26cb96'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:49.076739 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1ac6598e-c321-4723-9739-c263ad532242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b24f1542cddf4fc9aa8003952e26cb96'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:49.111299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1193/2387] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:17:48 2022] DELETE /identity/v3/users/b24f1542cddf4fc9aa8003952e26cb96 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:49.595440 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780994c5-f40b-4871-ab12-17ff7e0342f4 service swift] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:49.617975 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780994c5-f40b-4871-ab12-17ff7e0342f4 service swift] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:49.724251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26356d98-db16-4d94-93b5-96c49873cf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:49.725821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26356d98-db16-4d94-93b5-96c49873cf88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:49.726717 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26356d98-db16-4d94-93b5-96c49873cf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:49.727237 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26356d98-db16-4d94-93b5-96c49873cf88 admin admin] PATH_INFO: `/v3/projects/f124871e909842f4b74ef9fb483f5dd9` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:49.729469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26356d98-db16-4d94-93b5-96c49873cf88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f124871e909842f4b74ef9fb483f5dd9)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:49.734912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26356d98-db16-4d94-93b5-96c49873cf88 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:49.825800 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-26356d98-db16-4d94-93b5-96c49873cf88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f124871e909842f4b74ef9fb483f5dd9', 'request_id': 'req-26356d98-db16-4d94-93b5-96c49873cf88'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:49.835288 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-26356d98-db16-4d94-93b5-96c49873cf88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f124871e909842f4b74ef9fb483f5dd9', 'request_id': 'req-26356d98-db16-4d94-93b5-96c49873cf88'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:49.848847 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-26356d98-db16-4d94-93b5-96c49873cf88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f124871e909842f4b74ef9fb483f5dd9', 'request_id': 'req-26356d98-db16-4d94-93b5-96c49873cf88'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:49.874577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-26356d98-db16-4d94-93b5-96c49873cf88 admin admin] The token cache is being invalidate because project f124871e909842f4b74ef9fb483f5dd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:17:49.876264 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-26356d98-db16-4d94-93b5-96c49873cf88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:49.878179 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-26356d98-db16-4d94-93b5-96c49873cf88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:17:49.885298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1195/2388] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:17:49 2022] DELETE /identity/v3/projects/f124871e909842f4b74ef9fb483f5dd9 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:49.887329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b01963-4ed5-49dd-b7be-68624af8b04b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:49.890140 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b01963-4ed5-49dd-b7be-68624af8b04b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:49.890140 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b01963-4ed5-49dd-b7be-68624af8b04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:49.928559 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-20b01963-4ed5-49dd-b7be-68624af8b04b None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:49.930690 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-20b01963-4ed5-49dd-b7be-68624af8b04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:50.109424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1194/2389] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:50.122262 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac8f90-5c72-4863-8869-fa830030b701 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:50.122603 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac8f90-5c72-4863-8869-fa830030b701 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:50.122889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac8f90-5c72-4863-8869-fa830030b701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:50.156530 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-82ac8f90-5c72-4863-8869-fa830030b701 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:50.158384 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-82ac8f90-5c72-4863-8869-fa830030b701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:50.170714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1196/2390] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:50.175409 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b01963-4ed5-49dd-b7be-68624af8b04b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:50.193493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b01963-4ed5-49dd-b7be-68624af8b04b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:50.198962 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600f58a-a804-438e-91ec-5c1ed51bccf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:50.199757 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600f58a-a804-438e-91ec-5c1ed51bccf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:50.200197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600f58a-a804-438e-91ec-5c1ed51bccf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:50.200555 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600f58a-a804-438e-91ec-5c1ed51bccf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:50.201646 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600f58a-a804-438e-91ec-5c1ed51bccf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:17:50.202025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600f58a-a804-438e-91ec-5c1ed51bccf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:50.204912 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600f58a-a804-438e-91ec-5c1ed51bccf2 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:50.262158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1195/2391] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:17:50.262448 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac8f90-5c72-4863-8869-fa830030b701 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:50.276656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac8f90-5c72-4863-8869-fa830030b701 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:50.283130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb894a6d-42b2-419e-9981-282b39bb486f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:50.284139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb894a6d-42b2-419e-9981-282b39bb486f admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:50.284421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb894a6d-42b2-419e-9981-282b39bb486f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:50.284768 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb894a6d-42b2-419e-9981-282b39bb486f admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:50.285448 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb894a6d-42b2-419e-9981-282b39bb486f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:50.288081 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb894a6d-42b2-419e-9981-282b39bb486f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:50.330341 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1197/2392] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:50 2022] POST /identity/v3/projects => generated 369 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:50.334534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600f58a-a804-438e-91ec-5c1ed51bccf2 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:50.351070 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600f58a-a804-438e-91ec-5c1ed51bccf2 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:50.412270 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:50.443303 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372421de-4ae6-4c45-9c55-9198b4e48ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:50.443970 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372421de-4ae6-4c45-9c55-9198b4e48ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:50.444372 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372421de-4ae6-4c45-9c55-9198b4e48ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:50.444649 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372421de-4ae6-4c45-9c55-9198b4e48ab3 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:50.445312 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372421de-4ae6-4c45-9c55-9198b4e48ab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:50.447446 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372421de-4ae6-4c45-9c55-9198b4e48ab3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:50.508099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1196/2393] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:50 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:50.509115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb894a6d-42b2-419e-9981-282b39bb486f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:50.539802 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb894a6d-42b2-419e-9981-282b39bb486f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:50.553240 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9991794-d005-43c5-b011-ccf65194280c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:50.553643 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9991794-d005-43c5-b011-ccf65194280c admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:50.553909 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9991794-d005-43c5-b011-ccf65194280c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:50.554235 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9991794-d005-43c5-b011-ccf65194280c admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:50.554951 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9991794-d005-43c5-b011-ccf65194280c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:50.557448 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9991794-d005-43c5-b011-ccf65194280c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:50.584452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1198/2394] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:17:50.589259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372421de-4ae6-4c45-9c55-9198b4e48ab3 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:50.601869 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372421de-4ae6-4c45-9c55-9198b4e48ab3 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:50.612098 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e87e8-7293-43cf-89af-d3059c897625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:50.612877 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e87e8-7293-43cf-89af-d3059c897625 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:50.613168 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e87e8-7293-43cf-89af-d3059c897625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:50.613443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e87e8-7293-43cf-89af-d3059c897625 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:50.614118 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e87e8-7293-43cf-89af-d3059c897625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:50.616563 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e87e8-7293-43cf-89af-d3059c897625 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:50.631537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1197/2395] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:50.636069 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9991794-d005-43c5-b011-ccf65194280c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:50.649300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9991794-d005-43c5-b011-ccf65194280c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:50.655045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158b6bc-ae91-4df9-a28e-dbf4873cfc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:50.655725 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158b6bc-ae91-4df9-a28e-dbf4873cfc83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:50.656059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158b6bc-ae91-4df9-a28e-dbf4873cfc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:50.656980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158b6bc-ae91-4df9-a28e-dbf4873cfc83 admin admin] PATH_INFO: `/v3/projects/9719107f875340abbc26a6a2d04f8937/users/6c4efd4078704bd8814ce91a2a23bd66/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:50.684588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158b6bc-ae91-4df9-a28e-dbf4873cfc83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9719107f875340abbc26a6a2d04f8937, user_id=6c4efd4078704bd8814ce91a2a23bd66, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:50.686195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158b6bc-ae91-4df9-a28e-dbf4873cfc83 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:50.701539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:50.710697 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1199/2396] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:50 2022] PUT /identity/v3/projects/9719107f875340abbc26a6a2d04f8937/users/6c4efd4078704bd8814ce91a2a23bd66/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:50.723457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d9fa0-5e75-48fb-8ee0-c714f2233cac None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:50.723806 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d9fa0-5e75-48fb-8ee0-c714f2233cac None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:50.724026 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d9fa0-5e75-48fb-8ee0-c714f2233cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:50.789350 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9a2d9fa0-5e75-48fb-8ee0-c714f2233cac None None] MFA Rules not processed for user `6c4efd4078704bd8814ce91a2a23bd66`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:50.789350 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9a2d9fa0-5e75-48fb-8ee0-c714f2233cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:50.991104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1198/2397] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:51.020857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158b6bc-ae91-4df9-a28e-dbf4873cfc83 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:51.023192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8158b6bc-ae91-4df9-a28e-dbf4873cfc83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:51.165964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158b6bc-ae91-4df9-a28e-dbf4873cfc83 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:51.169882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924834cf-37ab-4081-a394-d55ba3215abc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50ef7db27744c7ebd2ec95cecc9d1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:51.170740 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924834cf-37ab-4081-a394-d55ba3215abc service glance] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:51.170911 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924834cf-37ab-4081-a394-d55ba3215abc service glance] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:51.171068 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924834cf-37ab-4081-a394-d55ba3215abc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:51.204874 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924834cf-37ab-4081-a394-d55ba3215abc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:51.208474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924834cf-37ab-4081-a394-d55ba3215abc service glance] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:51.224764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1200/2398] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:17:51.702769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb0dc5-16bd-4514-a977-a80c0d15c244 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:51.703191 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb0dc5-16bd-4514-a977-a80c0d15c244 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:51.704022 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb0dc5-16bd-4514-a977-a80c0d15c244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:51.754238 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1ffb0dc5-16bd-4514-a977-a80c0d15c244 None None] MFA Rules not processed for user `e0eaab966616458592a89cdef9cf63c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:51.768478 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1ffb0dc5-16bd-4514-a977-a80c0d15c244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:51.898775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1199/2399] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:17:51.904776 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924834cf-37ab-4081-a394-d55ba3215abc service glance] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:51.907330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-924834cf-37ab-4081-a394-d55ba3215abc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:52.020081 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924834cf-37ab-4081-a394-d55ba3215abc service glance] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:52.028857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a528f-49c8-4f92-ae3e-bd15201c979b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:52.029891 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a528f-49c8-4f92-ae3e-bd15201c979b admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:52.030472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a528f-49c8-4f92-ae3e-bd15201c979b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:52.030834 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a528f-49c8-4f92-ae3e-bd15201c979b admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:52.032164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a528f-49c8-4f92-ae3e-bd15201c979b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:52.034248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a528f-49c8-4f92-ae3e-bd15201c979b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:52.082182 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1201/2400] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:17:51 2022] POST /identity/v3/projects => generated 373 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:52.084889 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb0dc5-16bd-4514-a977-a80c0d15c244 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:52.102831 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb0dc5-16bd-4514-a977-a80c0d15c244 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:52.174460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:52.198086 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb2a22-7750-465d-b054-e45f06800bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:52.199177 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb2a22-7750-465d-b054-e45f06800bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:52.199613 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb2a22-7750-465d-b054-e45f06800bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:52.200686 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb2a22-7750-465d-b054-e45f06800bd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:52.202151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb2a22-7750-465d-b054-e45f06800bd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:52.205599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb2a22-7750-465d-b054-e45f06800bd4 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:52.251754 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1200/2401] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:17:52 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:17:52.255704 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a528f-49c8-4f92-ae3e-bd15201c979b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:52.276880 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a528f-49c8-4f92-ae3e-bd15201c979b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:52.284452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29753d-6385-4e37-a143-1bab5ae7efa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:52.285183 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29753d-6385-4e37-a143-1bab5ae7efa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:52.285507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29753d-6385-4e37-a143-1bab5ae7efa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:52.285844 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29753d-6385-4e37-a143-1bab5ae7efa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:52.286587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29753d-6385-4e37-a143-1bab5ae7efa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:52.288389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29753d-6385-4e37-a143-1bab5ae7efa6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:52.309151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1202/2402] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:17:52.320544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb2a22-7750-465d-b054-e45f06800bd4 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:52.342787 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb2a22-7750-465d-b054-e45f06800bd4 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:52.353494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b57b8-ef21-4221-8cc0-57541180895c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:52.354206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b57b8-ef21-4221-8cc0-57541180895c admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:52.354511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b57b8-ef21-4221-8cc0-57541180895c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:52.354808 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b57b8-ef21-4221-8cc0-57541180895c admin admin] PATH_INFO: `/v3/projects/6b00e1d8e21d4306b25c7361c3f9e305/users/20c0e7cd6c6043a39cc712176801493f/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:52.379250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b57b8-ef21-4221-8cc0-57541180895c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b00e1d8e21d4306b25c7361c3f9e305, user_id=20c0e7cd6c6043a39cc712176801493f, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:52.382248 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b57b8-ef21-4221-8cc0-57541180895c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:52.403862 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1201/2403] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:52 2022] PUT /identity/v3/projects/6b00e1d8e21d4306b25c7361c3f9e305/users/20c0e7cd6c6043a39cc712176801493f/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:52.410843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29753d-6385-4e37-a143-1bab5ae7efa6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:52.427893 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29753d-6385-4e37-a143-1bab5ae7efa6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:52.518589 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63953a80-1fcc-4782-8c47-4117aa6d3073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:52.519251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63953a80-1fcc-4782-8c47-4117aa6d3073 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:52.519516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63953a80-1fcc-4782-8c47-4117aa6d3073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:52.519788 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63953a80-1fcc-4782-8c47-4117aa6d3073 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:52.520458 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63953a80-1fcc-4782-8c47-4117aa6d3073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:52.522118 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63953a80-1fcc-4782-8c47-4117aa6d3073 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:52.536561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1203/2404] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:17:52.541208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b57b8-ef21-4221-8cc0-57541180895c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:52.566393 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b57b8-ef21-4221-8cc0-57541180895c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:52.572470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16387002-2529-45fc-aded-f5a38bb5ffaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:52.573181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16387002-2529-45fc-aded-f5a38bb5ffaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:52.574742 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16387002-2529-45fc-aded-f5a38bb5ffaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:52.575309 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16387002-2529-45fc-aded-f5a38bb5ffaf admin admin] PATH_INFO: `/v3/projects/6b00e1d8e21d4306b25c7361c3f9e305/users/20c0e7cd6c6043a39cc712176801493f/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:52.580011 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16387002-2529-45fc-aded-f5a38bb5ffaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b00e1d8e21d4306b25c7361c3f9e305, user_id=20c0e7cd6c6043a39cc712176801493f, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:52.583022 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16387002-2529-45fc-aded-f5a38bb5ffaf admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:52.595438 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1202/2405] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:17:52 2022] PUT /identity/v3/projects/6b00e1d8e21d4306b25c7361c3f9e305/users/20c0e7cd6c6043a39cc712176801493f/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:17:58.975225 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e9447-94ba-49d7-96c5-4c350ffcef4f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:58.975805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e9447-94ba-49d7-96c5-4c350ffcef4f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:58.975805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e9447-94ba-49d7-96c5-4c350ffcef4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:59.005865 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:17:59.016060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2b7e9447-94ba-49d7-96c5-4c350ffcef4f None None] MFA Rules not processed for user `20c0e7cd6c6043a39cc712176801493f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:17:59.018411 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2b7e9447-94ba-49d7-96c5-4c350ffcef4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:59.112259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1204/2406] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:17:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:17:59.126642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16387002-2529-45fc-aded-f5a38bb5ffaf admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:17:59.128415 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-16387002-2529-45fc-aded-f5a38bb5ffaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:17:59.276684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16387002-2529-45fc-aded-f5a38bb5ffaf admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:17:59.285516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ca906-e9ce-4af5-8e66-5867f8d5095b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:17:59.286295 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ca906-e9ce-4af5-8e66-5867f8d5095b service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:17:59.286731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ca906-e9ce-4af5-8e66-5867f8d5095b service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:17:59.286938 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ca906-e9ce-4af5-8e66-5867f8d5095b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:17:59.402298 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ca906-e9ce-4af5-8e66-5867f8d5095b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:17:59.414750 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ca906-e9ce-4af5-8e66-5867f8d5095b service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:17:59.444747 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1203/2407] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:17:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:01.000790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8580de0-350f-4337-9b62-8c665076f5f0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:01.001401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8580de0-350f-4337-9b62-8c665076f5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:01.001401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8580de0-350f-4337-9b62-8c665076f5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:01.022696 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d8580de0-350f-4337-9b62-8c665076f5f0 None None] MFA Rules not processed for user `e0eaab966616458592a89cdef9cf63c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:01.023991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d8580de0-350f-4337-9b62-8c665076f5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:01.032890 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1205/2408] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:18:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:09.438008 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=79,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:11.017089 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=32 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:26.109832 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ca906-e9ce-4af5-8e66-5867f8d5095b service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:26.111517 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-274ca906-e9ce-4af5-8e66-5867f8d5095b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:26.190720 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ca906-e9ce-4af5-8e66-5867f8d5095b service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:26.195535 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff3b97-8e23-4438-bb98-e88b483c895b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:26.196276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff3b97-8e23-4438-bb98-e88b483c895b service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:26.196659 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff3b97-8e23-4438-bb98-e88b483c895b service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:26.196972 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff3b97-8e23-4438-bb98-e88b483c895b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:26.199611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6aff3b97-8e23-4438-bb98-e88b483c895b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:26.313286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff3b97-8e23-4438-bb98-e88b483c895b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:26.315536 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff3b97-8e23-4438-bb98-e88b483c895b service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:26.332813 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1204/2409] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:36.322652 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=50 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:42.024510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8580de0-350f-4337-9b62-8c665076f5f0 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:42.025970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d8580de0-350f-4337-9b62-8c665076f5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:42.102723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8580de0-350f-4337-9b62-8c665076f5f0 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:42.107401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f5819-4057-4693-b766-dfc363c734e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:42.108039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f5819-4057-4693-b766-dfc363c734e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:42.108320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f5819-4057-4693-b766-dfc363c734e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:42.108592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f5819-4057-4693-b766-dfc363c734e6 admin admin] PATH_INFO: `/v3/users/6077450b6a05477fb3eb3af7ad3889bc` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:42.109817 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f5819-4057-4693-b766-dfc363c734e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6077450b6a05477fb3eb3af7ad3889bc)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:42.111990 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f5819-4057-4693-b766-dfc363c734e6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:42.163804 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2e9f5819-4057-4693-b766-dfc363c734e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6077450b6a05477fb3eb3af7ad3889bc'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:42.171039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2e9f5819-4057-4693-b766-dfc363c734e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6077450b6a05477fb3eb3af7ad3889bc'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:42.186178 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2e9f5819-4057-4693-b766-dfc363c734e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6077450b6a05477fb3eb3af7ad3889bc'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:42.186962 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2e9f5819-4057-4693-b766-dfc363c734e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6077450b6a05477fb3eb3af7ad3889bc'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:42.196756 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2e9f5819-4057-4693-b766-dfc363c734e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6077450b6a05477fb3eb3af7ad3889bc'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:42.203503 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1206/2410] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:18:42 2022] DELETE /identity/v3/users/6077450b6a05477fb3eb3af7ad3889bc => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:42.213806 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406173f1-9098-4e81-952c-47c90ce50008 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:42.214454 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406173f1-9098-4e81-952c-47c90ce50008 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:42.214454 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406173f1-9098-4e81-952c-47c90ce50008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:42.234616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-406173f1-9098-4e81-952c-47c90ce50008 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:18:42.235866 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-406173f1-9098-4e81-952c-47c90ce50008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:42.300466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1205/2411] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:18:42.312278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f5819-4057-4693-b766-dfc363c734e6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:42.313858 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2e9f5819-4057-4693-b766-dfc363c734e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:42.376914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f5819-4057-4693-b766-dfc363c734e6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:42.381537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9460a5-b1c2-4c99-aafa-a952f43a3d33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:42.382143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9460a5-b1c2-4c99-aafa-a952f43a3d33 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:42.382429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9460a5-b1c2-4c99-aafa-a952f43a3d33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:42.382730 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9460a5-b1c2-4c99-aafa-a952f43a3d33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:42.392347 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9460a5-b1c2-4c99-aafa-a952f43a3d33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:42.393776 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9460a5-b1c2-4c99-aafa-a952f43a3d33 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:42.404474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1207/2412] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:18:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:18:42.711661 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406173f1-9098-4e81-952c-47c90ce50008 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:18:42.713966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-406173f1-9098-4e81-952c-47c90ce50008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:18:42.734238 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406173f1-9098-4e81-952c-47c90ce50008 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:18:42.743067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac0015-5965-4345-89f1-8b0a8494d2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:18:42.744071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac0015-5965-4345-89f1-8b0a8494d2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:18:42.744502 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac0015-5965-4345-89f1-8b0a8494d2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:18:42.744966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac0015-5965-4345-89f1-8b0a8494d2fb admin admin] PATH_INFO: `/v3/projects/86604c9fb44c467098f915ff13b020f2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:18:42.746697 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac0015-5965-4345-89f1-8b0a8494d2fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=86604c9fb44c467098f915ff13b020f2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:18:42.751187 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac0015-5965-4345-89f1-8b0a8494d2fb admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:18:42.806756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-deac0015-5965-4345-89f1-8b0a8494d2fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86604c9fb44c467098f915ff13b020f2', 'request_id': 'req-deac0015-5965-4345-89f1-8b0a8494d2fb'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:42.815037 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-deac0015-5965-4345-89f1-8b0a8494d2fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86604c9fb44c467098f915ff13b020f2', 'request_id': 'req-deac0015-5965-4345-89f1-8b0a8494d2fb'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:42.822150 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-deac0015-5965-4345-89f1-8b0a8494d2fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86604c9fb44c467098f915ff13b020f2', 'request_id': 'req-deac0015-5965-4345-89f1-8b0a8494d2fb'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:42.833927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-deac0015-5965-4345-89f1-8b0a8494d2fb admin admin] The token cache is being invalidate because project 86604c9fb44c467098f915ff13b020f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:18:42.834246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-deac0015-5965-4345-89f1-8b0a8494d2fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:42.834943 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-deac0015-5965-4345-89f1-8b0a8494d2fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:18:42.836469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1206/2413] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:18:42 2022] DELETE /identity/v3/projects/86604c9fb44c467098f915ff13b020f2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:18:52.400003 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:18:52.830470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:19:21.245972 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9460a5-b1c2-4c99-aafa-a952f43a3d33 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:21.247693 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bb9460a5-b1c2-4c99-aafa-a952f43a3d33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:21.327835 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9460a5-b1c2-4c99-aafa-a952f43a3d33 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:21.382712 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0c7f9-9e83-4af5-83e2-7dfa45d3eece service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:21.383344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0c7f9-9e83-4af5-83e2-7dfa45d3eece service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:21.383692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0c7f9-9e83-4af5-83e2-7dfa45d3eece service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:21.384000 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0c7f9-9e83-4af5-83e2-7dfa45d3eece service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:21.385933 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5fd0c7f9-9e83-4af5-83e2-7dfa45d3eece service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:21.467287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0c7f9-9e83-4af5-83e2-7dfa45d3eece service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:21.468801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0c7f9-9e83-4af5-83e2-7dfa45d3eece service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:21.482706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1208/2414] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:19:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:19:21.787989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac0015-5965-4345-89f1-8b0a8494d2fb admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:21.789500 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-deac0015-5965-4345-89f1-8b0a8494d2fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:21.802960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac0015-5965-4345-89f1-8b0a8494d2fb admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:21.814224 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8076b00-1b09-4981-bd43-bee67ec87592 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:21.815294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8076b00-1b09-4981-bd43-bee67ec87592 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:21.815786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8076b00-1b09-4981-bd43-bee67ec87592 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:21.816359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8076b00-1b09-4981-bd43-bee67ec87592 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:21.819421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e8076b00-1b09-4981-bd43-bee67ec87592 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:21.843204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8076b00-1b09-4981-bd43-bee67ec87592 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:21.844765 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8076b00-1b09-4981-bd43-bee67ec87592 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:21.858011 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1207/2415] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:19:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:19:26.923101 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0c7f9-9e83-4af5-83e2-7dfa45d3eece service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:26.924563 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5fd0c7f9-9e83-4af5-83e2-7dfa45d3eece service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:26.936937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0c7f9-9e83-4af5-83e2-7dfa45d3eece service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:26.940709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4d52e-1017-46b4-b4fc-6d9865d0c707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:26.941310 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4d52e-1017-46b4-b4fc-6d9865d0c707 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:26.941585 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4d52e-1017-46b4-b4fc-6d9865d0c707 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:26.941852 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4d52e-1017-46b4-b4fc-6d9865d0c707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:26.943853 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-acb4d52e-1017-46b4-b4fc-6d9865d0c707 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:27.023201 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:19:27.028347 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4d52e-1017-46b4-b4fc-6d9865d0c707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:27.031845 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4d52e-1017-46b4-b4fc-6d9865d0c707 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:27.049728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1209/2416] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:19:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:19:31.851605 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=6 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:19:36.077682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a07f64-c461-459c-a759-03926dfbb552 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:36.078314 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a07f64-c461-459c-a759-03926dfbb552 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:36.078610 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a07f64-c461-459c-a759-03926dfbb552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:36.102091 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-a5a07f64-c461-459c-a759-03926dfbb552 None None] MFA Rules not processed for user `95ac6a289fb345d3a98871f5e11f9414`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:19:36.103460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a5a07f64-c461-459c-a759-03926dfbb552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:36.111999 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1208/2417] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:19:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:19:36.479329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4d52e-1017-46b4-b4fc-6d9865d0c707 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:36.480967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-acb4d52e-1017-46b4-b4fc-6d9865d0c707 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:36.554000 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4d52e-1017-46b4-b4fc-6d9865d0c707 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:36.558163 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa4d91-f15f-4898-9088-0d31d0846e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:36.559322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa4d91-f15f-4898-9088-0d31d0846e81 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:36.559690 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa4d91-f15f-4898-9088-0d31d0846e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:36.560011 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa4d91-f15f-4898-9088-0d31d0846e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:36.562112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e7aa4d91-f15f-4898-9088-0d31d0846e81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:36.573152 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa4d91-f15f-4898-9088-0d31d0846e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:36.575534 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa4d91-f15f-4898-9088-0d31d0846e81 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:36.584847 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1210/2418] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:19:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:19:36.962575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a07f64-c461-459c-a759-03926dfbb552 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:36.964189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a5a07f64-c461-459c-a759-03926dfbb552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:37.068049 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a07f64-c461-459c-a759-03926dfbb552 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:37.073556 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb8a18-a277-43cb-bffd-41ad3f7075e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034d75ab747e4c0cab040e47c6a6236e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:37.075186 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb8a18-a277-43cb-bffd-41ad3f7075e9 service placement] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:37.075554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb8a18-a277-43cb-bffd-41ad3f7075e9 service placement] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:37.075964 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb8a18-a277-43cb-bffd-41ad3f7075e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:37.078199 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cfcb8a18-a277-43cb-bffd-41ad3f7075e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:37.091875 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb8a18-a277-43cb-bffd-41ad3f7075e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:37.093712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb8a18-a277-43cb-bffd-41ad3f7075e9 service placement] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:37.109064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1209/2419] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:19:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:19:38.836847 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa4d91-f15f-4898-9088-0d31d0846e81 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:38.838398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e7aa4d91-f15f-4898-9088-0d31d0846e81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:38.916011 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa4d91-f15f-4898-9088-0d31d0846e81 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:38.921608 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a509e71-a96e-4aed-b7a8-6eed396efc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:38.922299 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a509e71-a96e-4aed-b7a8-6eed396efc05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:38.922601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a509e71-a96e-4aed-b7a8-6eed396efc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:38.922906 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a509e71-a96e-4aed-b7a8-6eed396efc05 admin admin] PATH_INFO: `/v3/users/6c4efd4078704bd8814ce91a2a23bd66` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:38.924131 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a509e71-a96e-4aed-b7a8-6eed396efc05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c4efd4078704bd8814ce91a2a23bd66)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:38.926140 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a509e71-a96e-4aed-b7a8-6eed396efc05 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:38.991196 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3a509e71-a96e-4aed-b7a8-6eed396efc05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c4efd4078704bd8814ce91a2a23bd66'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:39.000412 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3a509e71-a96e-4aed-b7a8-6eed396efc05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c4efd4078704bd8814ce91a2a23bd66'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:39.015213 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3a509e71-a96e-4aed-b7a8-6eed396efc05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c4efd4078704bd8814ce91a2a23bd66'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:39.019118 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3a509e71-a96e-4aed-b7a8-6eed396efc05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c4efd4078704bd8814ce91a2a23bd66'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:39.029403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3a509e71-a96e-4aed-b7a8-6eed396efc05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c4efd4078704bd8814ce91a2a23bd66'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:39.037746 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1211/2420] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:19:38 2022] DELETE /identity/v3/users/6c4efd4078704bd8814ce91a2a23bd66 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:19:39.042430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e4c04-f3f0-4c4e-8bfe-0a0f655c2f27 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:39.043208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e4c04-f3f0-4c4e-8bfe-0a0f655c2f27 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:39.043208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e4c04-f3f0-4c4e-8bfe-0a0f655c2f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:39.067692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-594e4c04-f3f0-4c4e-8bfe-0a0f655c2f27 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:19:39.068938 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-594e4c04-f3f0-4c4e-8bfe-0a0f655c2f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:39.144955 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1210/2421] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:19:39.155995 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a509e71-a96e-4aed-b7a8-6eed396efc05 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:39.157729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3a509e71-a96e-4aed-b7a8-6eed396efc05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:39.194152 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:19:39.281618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a509e71-a96e-4aed-b7a8-6eed396efc05 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:39.287583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37774e-eacb-4303-bacd-31a5ce19e88c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:39.288271 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37774e-eacb-4303-bacd-31a5ce19e88c service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:39.288559 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37774e-eacb-4303-bacd-31a5ce19e88c service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:39.288832 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37774e-eacb-4303-bacd-31a5ce19e88c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:39.303051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37774e-eacb-4303-bacd-31a5ce19e88c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:39.304687 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37774e-eacb-4303-bacd-31a5ce19e88c service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:39.321932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1212/2422] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:19:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:19:39.665991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e4c04-f3f0-4c4e-8bfe-0a0f655c2f27 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:39.668124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-594e4c04-f3f0-4c4e-8bfe-0a0f655c2f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:39.681769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e4c04-f3f0-4c4e-8bfe-0a0f655c2f27 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:39.689327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bed2c0-e120-4a0d-85f3-0c6f127c4d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:39.690298 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bed2c0-e120-4a0d-85f3-0c6f127c4d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:39.690754 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bed2c0-e120-4a0d-85f3-0c6f127c4d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:39.692084 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bed2c0-e120-4a0d-85f3-0c6f127c4d2e admin admin] PATH_INFO: `/v3/projects/9719107f875340abbc26a6a2d04f8937` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:39.693588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bed2c0-e120-4a0d-85f3-0c6f127c4d2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9719107f875340abbc26a6a2d04f8937)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:39.695760 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bed2c0-e120-4a0d-85f3-0c6f127c4d2e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:39.764284 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f9bed2c0-e120-4a0d-85f3-0c6f127c4d2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9719107f875340abbc26a6a2d04f8937', 'request_id': 'req-f9bed2c0-e120-4a0d-85f3-0c6f127c4d2e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:39.776215 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f9bed2c0-e120-4a0d-85f3-0c6f127c4d2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9719107f875340abbc26a6a2d04f8937', 'request_id': 'req-f9bed2c0-e120-4a0d-85f3-0c6f127c4d2e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:39.784195 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f9bed2c0-e120-4a0d-85f3-0c6f127c4d2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9719107f875340abbc26a6a2d04f8937', 'request_id': 'req-f9bed2c0-e120-4a0d-85f3-0c6f127c4d2e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:39.804727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f9bed2c0-e120-4a0d-85f3-0c6f127c4d2e admin admin] The token cache is being invalidate because project 9719107f875340abbc26a6a2d04f8937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:19:39.805316 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f9bed2c0-e120-4a0d-85f3-0c6f127c4d2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:39.806109 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f9bed2c0-e120-4a0d-85f3-0c6f127c4d2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:19:39.807948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1211/2423] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:19:39 2022] DELETE /identity/v3/projects/9719107f875340abbc26a6a2d04f8937 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:19:39.813422 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e9b5e-51b1-4f3a-8d7d-635ba1b0b7d2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:39.813788 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e9b5e-51b1-4f3a-8d7d-635ba1b0b7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:39.814230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e9b5e-51b1-4f3a-8d7d-635ba1b0b7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:39.836348 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c55e9b5e-51b1-4f3a-8d7d-635ba1b0b7d2 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:19:39.837782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c55e9b5e-51b1-4f3a-8d7d-635ba1b0b7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:39.910116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1213/2424] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:19:39.919301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a25cae-ad44-4a85-9ae0-317a387eba12 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:39.919653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a25cae-ad44-4a85-9ae0-317a387eba12 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:39.919930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a25cae-ad44-4a85-9ae0-317a387eba12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:39.935554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:19:39.943183 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-22a25cae-ad44-4a85-9ae0-317a387eba12 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:19:39.944498 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-22a25cae-ad44-4a85-9ae0-317a387eba12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:39.954088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1212/2425] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:19:39.958440 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e9b5e-51b1-4f3a-8d7d-635ba1b0b7d2 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:39.970855 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e9b5e-51b1-4f3a-8d7d-635ba1b0b7d2 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:39.979969 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471622f-f0f8-40b0-8ae9-2264637a8a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:39.981629 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471622f-f0f8-40b0-8ae9-2264637a8a10 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:39.982408 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471622f-f0f8-40b0-8ae9-2264637a8a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:39.983032 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471622f-f0f8-40b0-8ae9-2264637a8a10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:39.984122 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471622f-f0f8-40b0-8ae9-2264637a8a10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:19:39.984653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471622f-f0f8-40b0-8ae9-2264637a8a10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:39.987989 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471622f-f0f8-40b0-8ae9-2264637a8a10 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:40.015101 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1214/2426] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:19:40.019240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a25cae-ad44-4a85-9ae0-317a387eba12 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:40.035509 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a25cae-ad44-4a85-9ae0-317a387eba12 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:40.045458 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7f238-9a28-49b3-a725-fbc1707415a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:40.046546 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7f238-9a28-49b3-a725-fbc1707415a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:40.047438 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7f238-9a28-49b3-a725-fbc1707415a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:40.048207 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7f238-9a28-49b3-a725-fbc1707415a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:40.050257 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7f238-9a28-49b3-a725-fbc1707415a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:40.054607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7f238-9a28-49b3-a725-fbc1707415a6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:40.114568 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1213/2427] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:19:40 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:19:40.116442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471622f-f0f8-40b0-8ae9-2264637a8a10 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:40.128787 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471622f-f0f8-40b0-8ae9-2264637a8a10 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:40.209650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8c246-8189-4347-a409-51443fca42fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:40.210291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8c246-8189-4347-a409-51443fca42fe admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:40.210575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8c246-8189-4347-a409-51443fca42fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:40.210882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8c246-8189-4347-a409-51443fca42fe admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:40.211529 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8c246-8189-4347-a409-51443fca42fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:40.213528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8c246-8189-4347-a409-51443fca42fe admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:40.249919 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1215/2428] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:19:40 2022] POST /identity/v3/users => generated 288 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:19:40.255648 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7f238-9a28-49b3-a725-fbc1707415a6 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:40.270302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7f238-9a28-49b3-a725-fbc1707415a6 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:40.275726 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69244f6-e32d-4886-9868-ede3003bcc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:40.276486 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69244f6-e32d-4886-9868-ede3003bcc1d admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:40.276861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69244f6-e32d-4886-9868-ede3003bcc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:40.277227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69244f6-e32d-4886-9868-ede3003bcc1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:40.278233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69244f6-e32d-4886-9868-ede3003bcc1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:40.280071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69244f6-e32d-4886-9868-ede3003bcc1d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:40.291921 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1214/2429] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:19:40.296618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8c246-8189-4347-a409-51443fca42fe admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:40.313283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8c246-8189-4347-a409-51443fca42fe admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:40.319212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb2fe0-daf2-4cc8-a587-268b4e2561eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:40.319837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb2fe0-daf2-4cc8-a587-268b4e2561eb admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:40.320145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb2fe0-daf2-4cc8-a587-268b4e2561eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:40.320389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb2fe0-daf2-4cc8-a587-268b4e2561eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:40.321043 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb2fe0-daf2-4cc8-a587-268b4e2561eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:40.322724 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb2fe0-daf2-4cc8-a587-268b4e2561eb admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:40.335087 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1216/2430] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:19:40.341290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69244f6-e32d-4886-9868-ede3003bcc1d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:40.355164 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69244f6-e32d-4886-9868-ede3003bcc1d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:40.362267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e2bcb-480d-4972-861a-911b01b56c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:40.362267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e2bcb-480d-4972-861a-911b01b56c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:40.362797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e2bcb-480d-4972-861a-911b01b56c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:40.362797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e2bcb-480d-4972-861a-911b01b56c1b admin admin] PATH_INFO: `/v3/projects/5873c8fe7f6346d3aaf9e4d8a0da4d4e/users/b4b5e94108d14067a2f760c89964b4f4/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:40.381519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e2bcb-480d-4972-861a-911b01b56c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5873c8fe7f6346d3aaf9e4d8a0da4d4e, user_id=b4b5e94108d14067a2f760c89964b4f4, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:40.384276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e2bcb-480d-4972-861a-911b01b56c1b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:40.406289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1215/2431] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:19:40 2022] PUT /identity/v3/projects/5873c8fe7f6346d3aaf9e4d8a0da4d4e/users/b4b5e94108d14067a2f760c89964b4f4/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:19:40.417733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d5a10-afee-48f2-8740-3cd00e759dbf None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:40.418077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d5a10-afee-48f2-8740-3cd00e759dbf None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:40.418291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d5a10-afee-48f2-8740-3cd00e759dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:40.452901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-941d5a10-afee-48f2-8740-3cd00e759dbf None None] MFA Rules not processed for user `b4b5e94108d14067a2f760c89964b4f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:19:40.454151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-941d5a10-afee-48f2-8740-3cd00e759dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:40.474053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:19:40.553154 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1217/2432] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:19:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:19:40.568069 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e2bcb-480d-4972-861a-911b01b56c1b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:19:40.570072 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-356e2bcb-480d-4972-861a-911b01b56c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:40.648576 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e2bcb-480d-4972-861a-911b01b56c1b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:19:40.671598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14e4db-6551-4e28-874a-597f5157b800 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:19:40.673032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14e4db-6551-4e28-874a-597f5157b800 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:40.673482 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14e4db-6551-4e28-874a-597f5157b800 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:40.673753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14e4db-6551-4e28-874a-597f5157b800 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:40.697784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14e4db-6551-4e28-874a-597f5157b800 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:19:40.699587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14e4db-6551-4e28-874a-597f5157b800 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:19:40.711670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1216/2433] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:19:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:19:43.239636 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4725b-8c0f-4e9c-88c2-a49e1fdb5eec None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:19:43.241227 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4725b-8c0f-4e9c-88c2-a49e1fdb5eec None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:19:43.242118 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4725b-8c0f-4e9c-88c2-a49e1fdb5eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:19:43.266998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8cd4725b-8c0f-4e9c-88c2-a49e1fdb5eec None None] MFA Rules not processed for user `b4b5e94108d14067a2f760c89964b4f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:19:43.268270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8cd4725b-8c0f-4e9c-88c2-a49e1fdb5eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:19:43.275257 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1218/2434] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:19:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:19:50.706605 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=50,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:19:53.261308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=23 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:20:04.439073 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14e4db-6551-4e28-874a-597f5157b800 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:04.440853 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3a14e4db-6551-4e28-874a-597f5157b800 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:04.513992 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14e4db-6551-4e28-874a-597f5157b800 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:04.518160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb1a8c-0c36-434d-ba09-8b51fc48e874 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:04.518895 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb1a8c-0c36-434d-ba09-8b51fc48e874 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:04.519192 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb1a8c-0c36-434d-ba09-8b51fc48e874 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:04.519455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb1a8c-0c36-434d-ba09-8b51fc48e874 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:04.521329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-99eb1a8c-0c36-434d-ba09-8b51fc48e874 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:04.539023 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb1a8c-0c36-434d-ba09-8b51fc48e874 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:04.541576 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb1a8c-0c36-434d-ba09-8b51fc48e874 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:04.553349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1217/2435] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:20:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:20:10.115925 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4725b-8c0f-4e9c-88c2-a49e1fdb5eec None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:10.117477 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8cd4725b-8c0f-4e9c-88c2-a49e1fdb5eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:10.229046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4725b-8c0f-4e9c-88c2-a49e1fdb5eec None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:10.237983 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2774aa1-aacf-40d5-8715-b6f98ad5b106 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:10.237983 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2774aa1-aacf-40d5-8715-b6f98ad5b106 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:10.238308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2774aa1-aacf-40d5-8715-b6f98ad5b106 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:10.238509 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2774aa1-aacf-40d5-8715-b6f98ad5b106 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:10.240376 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d2774aa1-aacf-40d5-8715-b6f98ad5b106 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:10.256921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2774aa1-aacf-40d5-8715-b6f98ad5b106 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:10.258992 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2774aa1-aacf-40d5-8715-b6f98ad5b106 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:10.271145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1219/2436] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:20:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:20:14.548576 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=26 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:20:17.108948 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb1a8c-0c36-434d-ba09-8b51fc48e874 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:17.124257 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb1a8c-0c36-434d-ba09-8b51fc48e874 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:17.130646 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397916d-53d3-41b7-9f11-01cb2ca936dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:17.131548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397916d-53d3-41b7-9f11-01cb2ca936dd service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:17.131833 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397916d-53d3-41b7-9f11-01cb2ca936dd service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:17.132110 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397916d-53d3-41b7-9f11-01cb2ca936dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:17.133892 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5397916d-53d3-41b7-9f11-01cb2ca936dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:17.147851 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397916d-53d3-41b7-9f11-01cb2ca936dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:17.149506 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397916d-53d3-41b7-9f11-01cb2ca936dd service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:17.160880 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1218/2437] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:20:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:20:20.266280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=34 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:20:22.676582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2774aa1-aacf-40d5-8715-b6f98ad5b106 service nova] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:22.686366 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2774aa1-aacf-40d5-8715-b6f98ad5b106 service nova] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:22.689971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3d3fa-cb2f-4434-9146-9f58aa4e4187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:22.690560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3d3fa-cb2f-4434-9146-9f58aa4e4187 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:22.690866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3d3fa-cb2f-4434-9146-9f58aa4e4187 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:22.691140 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3d3fa-cb2f-4434-9146-9f58aa4e4187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:22.693665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e4e3d3fa-cb2f-4434-9146-9f58aa4e4187 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:22.756841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3d3fa-cb2f-4434-9146-9f58aa4e4187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:22.758262 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3d3fa-cb2f-4434-9146-9f58aa4e4187 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:22.767998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1220/2438] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:20:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:20:27.156029 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=6 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:20:32.763729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=30 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:20:37.366189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e6db8-8002-4de6-935f-e5d16be00de5 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:37.366789 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e6db8-8002-4de6-935f-e5d16be00de5 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:37.366789 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e6db8-8002-4de6-935f-e5d16be00de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:37.387313 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-528e6db8-8002-4de6-935f-e5d16be00de5 None None] MFA Rules not processed for user `b266573c02ab476da327907dcfed5583`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:20:37.388548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-528e6db8-8002-4de6-935f-e5d16be00de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:37.453838 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1219/2439] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:20:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:20:37.794045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3d3fa-cb2f-4434-9146-9f58aa4e4187 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:37.803209 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3d3fa-cb2f-4434-9146-9f58aa4e4187 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:37.806939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2316f73-d450-49be-92f6-5f69c56edfdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:37.807531 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2316f73-d450-49be-92f6-5f69c56edfdf service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:37.807806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2316f73-d450-49be-92f6-5f69c56edfdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:37.808079 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2316f73-d450-49be-92f6-5f69c56edfdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:37.809809 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e2316f73-d450-49be-92f6-5f69c56edfdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:37.820578 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2316f73-d450-49be-92f6-5f69c56edfdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:37.822041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2316f73-d450-49be-92f6-5f69c56edfdf service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:37.830012 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1221/2440] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:20:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:20:39.201588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd23515-f568-4c94-949d-47b0feb7fcc3 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:39.202056 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd23515-f568-4c94-949d-47b0feb7fcc3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:39.202147 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd23515-f568-4c94-949d-47b0feb7fcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:39.221818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1bd23515-f568-4c94-949d-47b0feb7fcc3 None None] MFA Rules not processed for user `ba58a4914ddf418db9e8b9dce5784cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:20:39.223161 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1bd23515-f568-4c94-949d-47b0feb7fcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:39.285671 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1220/2441] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 6 16:20:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:20:39.335967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2316f73-d450-49be-92f6-5f69c56edfdf service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:39.337384 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e2316f73-d450-49be-92f6-5f69c56edfdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:39.413383 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2316f73-d450-49be-92f6-5f69c56edfdf service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:39.418366 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a02e9-9e3f-4fa4-baf3-4d7df5cc8326 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ece176d412574790a13d03dcca08f91a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:39.418998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a02e9-9e3f-4fa4-baf3-4d7df5cc8326 service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:39.419282 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a02e9-9e3f-4fa4-baf3-4d7df5cc8326 service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:39.419558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a02e9-9e3f-4fa4-baf3-4d7df5cc8326 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:39.427819 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a02e9-9e3f-4fa4-baf3-4d7df5cc8326 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:39.429394 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a02e9-9e3f-4fa4-baf3-4d7df5cc8326 service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:39.437598 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1222/2442] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 6 16:20:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:20:40.021925 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd755e00-48d3-4621-ad88-856a947fa352 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:40.022211 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd755e00-48d3-4621-ad88-856a947fa352 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:40.022519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd755e00-48d3-4621-ad88-856a947fa352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:40.043208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-bd755e00-48d3-4621-ad88-856a947fa352 None None] MFA Rules not processed for user `ba58a4914ddf418db9e8b9dce5784cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:20:40.044565 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bd755e00-48d3-4621-ad88-856a947fa352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:40.051991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1221/2443] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 6 16:20:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:20:40.064354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a02e9-9e3f-4fa4-baf3-4d7df5cc8326 service swift] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:40.076725 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a02e9-9e3f-4fa4-baf3-4d7df5cc8326 service swift] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:40.081807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d78fd7-6595-4569-aad3-0bc8bdcdb4ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ece176d412574790a13d03dcca08f91a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:40.082406 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d78fd7-6595-4569-aad3-0bc8bdcdb4ce service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:40.082740 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d78fd7-6595-4569-aad3-0bc8bdcdb4ce service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:40.083011 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d78fd7-6595-4569-aad3-0bc8bdcdb4ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:40.092934 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d78fd7-6595-4569-aad3-0bc8bdcdb4ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:40.094578 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d78fd7-6595-4569-aad3-0bc8bdcdb4ce service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:40.105075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1223/2444] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 6 16:20:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:20:40.384544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebbe99-c267-40a0-8409-8f0fa87fcd39 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:40.385887 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebbe99-c267-40a0-8409-8f0fa87fcd39 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:40.386413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebbe99-c267-40a0-8409-8f0fa87fcd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:40.420460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-93ebbe99-c267-40a0-8409-8f0fa87fcd39 None None] MFA Rules not processed for user `ba58a4914ddf418db9e8b9dce5784cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:20:40.422359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-93ebbe99-c267-40a0-8409-8f0fa87fcd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:40.439436 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1222/2445] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 6 16:20:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:20:40.448614 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d78fd7-6595-4569-aad3-0bc8bdcdb4ce service swift] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:40.464165 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d78fd7-6595-4569-aad3-0bc8bdcdb4ce service swift] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:40.472457 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58fef5-1425-4e81-a736-9dad74cdf0cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ece176d412574790a13d03dcca08f91a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:40.473358 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58fef5-1425-4e81-a736-9dad74cdf0cf service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:40.474374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58fef5-1425-4e81-a736-9dad74cdf0cf service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:40.474954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58fef5-1425-4e81-a736-9dad74cdf0cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:40.495580 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58fef5-1425-4e81-a736-9dad74cdf0cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:40.497184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58fef5-1425-4e81-a736-9dad74cdf0cf service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:40.506244 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1224/2446] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 6 16:20:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:20:40.874336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994821f-3321-4868-b69d-a76c7940d0fd None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:40.874627 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994821f-3321-4868-b69d-a76c7940d0fd None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:40.874937 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994821f-3321-4868-b69d-a76c7940d0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:40.896330 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-2994821f-3321-4868-b69d-a76c7940d0fd None None] MFA Rules not processed for user `ba58a4914ddf418db9e8b9dce5784cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:20:40.897410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2994821f-3321-4868-b69d-a76c7940d0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:40.905042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1223/2447] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 6 16:20:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:20:40.918761 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58fef5-1425-4e81-a736-9dad74cdf0cf service swift] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:40.932001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58fef5-1425-4e81-a736-9dad74cdf0cf service swift] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:40.937315 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502511d8-cf05-4a66-9f15-207bab452b48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ece176d412574790a13d03dcca08f91a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:40.937923 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502511d8-cf05-4a66-9f15-207bab452b48 service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:40.938200 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502511d8-cf05-4a66-9f15-207bab452b48 service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:40.938493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502511d8-cf05-4a66-9f15-207bab452b48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:40.948952 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502511d8-cf05-4a66-9f15-207bab452b48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:40.950640 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502511d8-cf05-4a66-9f15-207bab452b48 service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:40.959985 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1225/2448] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 6 16:20:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 16:20:43.219577 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994821f-3321-4868-b69d-a76c7940d0fd None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:43.286019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994821f-3321-4868-b69d-a76c7940d0fd None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:43.291411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e6ebd-3ebb-4e8f-b7df-a149f46203e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:43.292061 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e6ebd-3ebb-4e8f-b7df-a149f46203e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:43.292340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e6ebd-3ebb-4e8f-b7df-a149f46203e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:43.292610 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e6ebd-3ebb-4e8f-b7df-a149f46203e4 admin admin] PATH_INFO: `/v3/users/b4b5e94108d14067a2f760c89964b4f4` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:43.293788 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e6ebd-3ebb-4e8f-b7df-a149f46203e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4b5e94108d14067a2f760c89964b4f4)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:43.295790 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e6ebd-3ebb-4e8f-b7df-a149f46203e4 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:43.332541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:20:43.347237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-3f0e6ebd-3ebb-4e8f-b7df-a149f46203e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4b5e94108d14067a2f760c89964b4f4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:20:43.355299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-3f0e6ebd-3ebb-4e8f-b7df-a149f46203e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4b5e94108d14067a2f760c89964b4f4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:20:43.367835 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-3f0e6ebd-3ebb-4e8f-b7df-a149f46203e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4b5e94108d14067a2f760c89964b4f4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:20:43.368581 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-3f0e6ebd-3ebb-4e8f-b7df-a149f46203e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4b5e94108d14067a2f760c89964b4f4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:20:43.369210 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-3f0e6ebd-3ebb-4e8f-b7df-a149f46203e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4b5e94108d14067a2f760c89964b4f4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:20:43.385534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1224/2449] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:20:43 2022] DELETE /identity/v3/users/b4b5e94108d14067a2f760c89964b4f4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:20:43.390273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daaddc5-c944-451c-b2de-642042f472a9 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:43.390840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daaddc5-c944-451c-b2de-642042f472a9 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:43.390964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daaddc5-c944-451c-b2de-642042f472a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:43.411381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8daaddc5-c944-451c-b2de-642042f472a9 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:20:43.412643 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8daaddc5-c944-451c-b2de-642042f472a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:43.477935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1226/2450] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:20:43.488129 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e6ebd-3ebb-4e8f-b7df-a149f46203e4 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:43.489564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3f0e6ebd-3ebb-4e8f-b7df-a149f46203e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:43.552455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e6ebd-3ebb-4e8f-b7df-a149f46203e4 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:43.557547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de0bd5-6d99-4c15-9695-7268de770c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:43.558611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de0bd5-6d99-4c15-9695-7268de770c07 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:43.559153 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de0bd5-6d99-4c15-9695-7268de770c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:43.559440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de0bd5-6d99-4c15-9695-7268de770c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:43.571710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de0bd5-6d99-4c15-9695-7268de770c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:43.573452 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de0bd5-6d99-4c15-9695-7268de770c07 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:43.590974 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1225/2451] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:20:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:20:43.797582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daaddc5-c944-451c-b2de-642042f472a9 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:20:43.800300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8daaddc5-c944-451c-b2de-642042f472a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:20:43.823186 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daaddc5-c944-451c-b2de-642042f472a9 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:20:43.831115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469154ec-e1b1-4d26-893d-0adf867d7509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:20:43.832214 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469154ec-e1b1-4d26-893d-0adf867d7509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:20:43.832715 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469154ec-e1b1-4d26-893d-0adf867d7509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:20:43.833202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469154ec-e1b1-4d26-893d-0adf867d7509 admin admin] PATH_INFO: `/v3/projects/5873c8fe7f6346d3aaf9e4d8a0da4d4e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:20:43.835391 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469154ec-e1b1-4d26-893d-0adf867d7509 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5873c8fe7f6346d3aaf9e4d8a0da4d4e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:20:43.839132 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469154ec-e1b1-4d26-893d-0adf867d7509 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:20:43.880200 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:20:43.897871 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-469154ec-e1b1-4d26-893d-0adf867d7509 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5873c8fe7f6346d3aaf9e4d8a0da4d4e', 'request_id': 'req-469154ec-e1b1-4d26-893d-0adf867d7509'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:20:43.904375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-469154ec-e1b1-4d26-893d-0adf867d7509 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5873c8fe7f6346d3aaf9e4d8a0da4d4e', 'request_id': 'req-469154ec-e1b1-4d26-893d-0adf867d7509'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:20:43.910482 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-469154ec-e1b1-4d26-893d-0adf867d7509 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5873c8fe7f6346d3aaf9e4d8a0da4d4e', 'request_id': 'req-469154ec-e1b1-4d26-893d-0adf867d7509'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:20:43.924912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-469154ec-e1b1-4d26-893d-0adf867d7509 admin admin] The token cache is being invalidate because project 5873c8fe7f6346d3aaf9e4d8a0da4d4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:20:43.925206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-469154ec-e1b1-4d26-893d-0adf867d7509 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:20:43.925854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-469154ec-e1b1-4d26-893d-0adf867d7509 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:20:43.927422 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1227/2452] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:20:43 2022] DELETE /identity/v3/projects/5873c8fe7f6346d3aaf9e4d8a0da4d4e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:20:53.581929 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=43,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:20:53.921707 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:21:30.741139 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de0bd5-6d99-4c15-9695-7268de770c07 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:30.742753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-07de0bd5-6d99-4c15-9695-7268de770c07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:30.826462 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de0bd5-6d99-4c15-9695-7268de770c07 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:30.903023 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012d663-b816-4591-8d62-f634e80d6551 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:30.903561 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012d663-b816-4591-8d62-f634e80d6551 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:30.903842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012d663-b816-4591-8d62-f634e80d6551 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:30.904114 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012d663-b816-4591-8d62-f634e80d6551 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:30.905931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0012d663-b816-4591-8d62-f634e80d6551 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:30.975757 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0012d663-b816-4591-8d62-f634e80d6551 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:30.977444 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0012d663-b816-4591-8d62-f634e80d6551 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:30.988352 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1226/2453] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:21:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:21:40.982837 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=84 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:21:50.908326 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469154ec-e1b1-4d26-893d-0adf867d7509 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:50.909842 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-469154ec-e1b1-4d26-893d-0adf867d7509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:50.931123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469154ec-e1b1-4d26-893d-0adf867d7509 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:50.938311 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77856aad-1a12-45ef-afb5-f662edff8a51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:50.938965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77856aad-1a12-45ef-afb5-f662edff8a51 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:50.939251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77856aad-1a12-45ef-afb5-f662edff8a51 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:50.939536 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77856aad-1a12-45ef-afb5-f662edff8a51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:50.941798 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-77856aad-1a12-45ef-afb5-f662edff8a51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:51.019094 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77856aad-1a12-45ef-afb5-f662edff8a51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:51.019094 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77856aad-1a12-45ef-afb5-f662edff8a51 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:51.031591 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1228/2454] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:21:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:21:56.842855 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012d663-b816-4591-8d62-f634e80d6551 service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:56.844380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0012d663-b816-4591-8d62-f634e80d6551 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:56.916339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77856aad-1a12-45ef-afb5-f662edff8a51 service nova] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:56.917890 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-77856aad-1a12-45ef-afb5-f662edff8a51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:56.927804 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012d663-b816-4591-8d62-f634e80d6551 service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:56.934813 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8acef5-f393-402e-aafc-59eefdc449b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034d75ab747e4c0cab040e47c6a6236e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:56.935374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8acef5-f393-402e-aafc-59eefdc449b2 service placement] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:56.935652 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8acef5-f393-402e-aafc-59eefdc449b2 service placement] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:56.935925 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8acef5-f393-402e-aafc-59eefdc449b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:56.937745 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-de8acef5-f393-402e-aafc-59eefdc449b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:56.958855 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8acef5-f393-402e-aafc-59eefdc449b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:56.960578 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8acef5-f393-402e-aafc-59eefdc449b2 service placement] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:56.973001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1227/2455] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:21:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:21:56.999811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77856aad-1a12-45ef-afb5-f662edff8a51 service nova] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:57.005884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a64cb-fcf5-442d-980d-527adfd669e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:57.007206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a64cb-fcf5-442d-980d-527adfd669e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:57.007520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a64cb-fcf5-442d-980d-527adfd669e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:57.008920 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a64cb-fcf5-442d-980d-527adfd669e8 admin admin] PATH_INFO: `/v3/users/b266573c02ab476da327907dcfed5583` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:57.011167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a64cb-fcf5-442d-980d-527adfd669e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b266573c02ab476da327907dcfed5583)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:57.015164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a64cb-fcf5-442d-980d-527adfd669e8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:57.092176 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bb6a64cb-fcf5-442d-980d-527adfd669e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b266573c02ab476da327907dcfed5583'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:57.102341 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bb6a64cb-fcf5-442d-980d-527adfd669e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b266573c02ab476da327907dcfed5583'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:57.112639 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bb6a64cb-fcf5-442d-980d-527adfd669e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b266573c02ab476da327907dcfed5583'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:57.113408 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bb6a64cb-fcf5-442d-980d-527adfd669e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b266573c02ab476da327907dcfed5583'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:57.123131 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bb6a64cb-fcf5-442d-980d-527adfd669e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b266573c02ab476da327907dcfed5583'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:57.132765 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1229/2456] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:21:56 2022] DELETE /identity/v3/users/b266573c02ab476da327907dcfed5583 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:21:57.184072 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8acef5-f393-402e-aafc-59eefdc449b2 service placement] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:57.203026 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8acef5-f393-402e-aafc-59eefdc449b2 service placement] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:57.285979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a2eaa-307d-43c9-8c44-dfe82c3b8d03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:57.286603 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a2eaa-307d-43c9-8c44-dfe82c3b8d03 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:57.286918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a2eaa-307d-43c9-8c44-dfe82c3b8d03 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:57.287189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a2eaa-307d-43c9-8c44-dfe82c3b8d03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:57.289020 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a32a2eaa-307d-43c9-8c44-dfe82c3b8d03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:57.379965 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a2eaa-307d-43c9-8c44-dfe82c3b8d03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:57.382299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a2eaa-307d-43c9-8c44-dfe82c3b8d03 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:57.397942 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a64cb-fcf5-442d-980d-527adfd669e8 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:21:57.399921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bb6a64cb-fcf5-442d-980d-527adfd669e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:21:57.402940 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1228/2457] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:21:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:21:57.428712 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:21:57.491603 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a64cb-fcf5-442d-980d-527adfd669e8 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:21:57.498697 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96db89-15b5-4b88-8f6d-5ad11660b769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:21:57.499407 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96db89-15b5-4b88-8f6d-5ad11660b769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:21:57.499710 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96db89-15b5-4b88-8f6d-5ad11660b769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:21:57.499950 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96db89-15b5-4b88-8f6d-5ad11660b769 admin admin] PATH_INFO: `/v3/projects/17e81ff9ac5c4468989eb31ba2130e0d` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:21:57.501181 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96db89-15b5-4b88-8f6d-5ad11660b769 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17e81ff9ac5c4468989eb31ba2130e0d)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:21:57.503269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96db89-15b5-4b88-8f6d-5ad11660b769 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:21:57.556352 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2e96db89-15b5-4b88-8f6d-5ad11660b769 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17e81ff9ac5c4468989eb31ba2130e0d', 'request_id': 'req-2e96db89-15b5-4b88-8f6d-5ad11660b769'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:57.563606 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2e96db89-15b5-4b88-8f6d-5ad11660b769 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17e81ff9ac5c4468989eb31ba2130e0d', 'request_id': 'req-2e96db89-15b5-4b88-8f6d-5ad11660b769'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:57.571190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2e96db89-15b5-4b88-8f6d-5ad11660b769 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17e81ff9ac5c4468989eb31ba2130e0d', 'request_id': 'req-2e96db89-15b5-4b88-8f6d-5ad11660b769'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:57.588540 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2e96db89-15b5-4b88-8f6d-5ad11660b769 admin admin] The token cache is being invalidate because project 17e81ff9ac5c4468989eb31ba2130e0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:21:57.588837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2e96db89-15b5-4b88-8f6d-5ad11660b769 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:57.589548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2e96db89-15b5-4b88-8f6d-5ad11660b769 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:21:57.591595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1230/2458] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:21:57 2022] DELETE /identity/v3/projects/17e81ff9ac5c4468989eb31ba2130e0d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:22:07.389402 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=88 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:22:07.583708 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=43,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:22:22.291299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02427cd9-ac28-4f6a-9bec-ab412c99235d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:22:22.291931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02427cd9-ac28-4f6a-9bec-ab412c99235d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:22:22.291931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02427cd9-ac28-4f6a-9bec-ab412c99235d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:22:22.323460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-02427cd9-ac28-4f6a-9bec-ab412c99235d None None] MFA Rules not processed for user `eda6839efaab4317a0eda1f47f1656a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:22:22.324692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-02427cd9-ac28-4f6a-9bec-ab412c99235d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:22.397877 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1229/2459] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:22:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:22:28.721304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96db89-15b5-4b88-8f6d-5ad11660b769 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:22:28.723843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2e96db89-15b5-4b88-8f6d-5ad11660b769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:28.952728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96db89-15b5-4b88-8f6d-5ad11660b769 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:22:28.964233 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3dc55-0aee-4bef-b1be-b804a27c9e4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:22:28.964909 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3dc55-0aee-4bef-b1be-b804a27c9e4c service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:22:28.965710 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3dc55-0aee-4bef-b1be-b804a27c9e4c service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:22:28.965710 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3dc55-0aee-4bef-b1be-b804a27c9e4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:22:28.967387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-67b3dc55-0aee-4bef-b1be-b804a27c9e4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:28.982710 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3dc55-0aee-4bef-b1be-b804a27c9e4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:22:28.984343 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3dc55-0aee-4bef-b1be-b804a27c9e4c service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:22:28.997427 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1231/2460] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:22:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:22:32.395471 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=37 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:22:38.991697 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=64 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:22:48.761029 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02427cd9-ac28-4f6a-9bec-ab412c99235d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:22:48.764026 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-02427cd9-ac28-4f6a-9bec-ab412c99235d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:48.885650 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02427cd9-ac28-4f6a-9bec-ab412c99235d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:22:48.891043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f598a8-e36e-4d34-b1f9-a41b2c98b348 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034d75ab747e4c0cab040e47c6a6236e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:22:48.891662 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f598a8-e36e-4d34-b1f9-a41b2c98b348 service placement] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:22:48.891938 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f598a8-e36e-4d34-b1f9-a41b2c98b348 service placement] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:22:48.892208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f598a8-e36e-4d34-b1f9-a41b2c98b348 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:22:48.894146 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f9f598a8-e36e-4d34-b1f9-a41b2c98b348 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:48.906497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f598a8-e36e-4d34-b1f9-a41b2c98b348 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:22:48.908392 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f598a8-e36e-4d34-b1f9-a41b2c98b348 service placement] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:22:48.919251 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1230/2461] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:22:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:22:58.915915 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=41 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:22:58.935466 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9528f-64f7-42c1-b237-963a30077130 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:22:58.936034 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9528f-64f7-42c1-b237-963a30077130 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:22:58.936034 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9528f-64f7-42c1-b237-963a30077130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:22:58.957970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-04d9528f-64f7-42c1-b237-963a30077130 None None] MFA Rules not processed for user `95ac6a289fb345d3a98871f5e11f9414`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:22:58.959207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-04d9528f-64f7-42c1-b237-963a30077130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:22:59.021610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1232/2462] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:22:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:23:09.014040 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=25 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:23:35.941930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66c4c9-adae-4fbc-974a-751b2c912625 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:35.942576 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66c4c9-adae-4fbc-974a-751b2c912625 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:35.942859 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66c4c9-adae-4fbc-974a-751b2c912625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:35.976439 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-3f66c4c9-adae-4fbc-974a-751b2c912625 None None] MFA Rules not processed for user `ba58a4914ddf418db9e8b9dce5784cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:23:35.977693 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3f66c4c9-adae-4fbc-974a-751b2c912625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:36.046801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1231/2463] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 6 16:23:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:23:36.101305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9528f-64f7-42c1-b237-963a30077130 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:36.102899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-04d9528f-64f7-42c1-b237-963a30077130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:36.188743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9528f-64f7-42c1-b237-963a30077130 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:36.194077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae64c0-8a84-481c-b186-11917929e03a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ece176d412574790a13d03dcca08f91a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:36.194738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae64c0-8a84-481c-b186-11917929e03a service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:36.195019 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae64c0-8a84-481c-b186-11917929e03a service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:36.195290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae64c0-8a84-481c-b186-11917929e03a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:36.207216 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae64c0-8a84-481c-b186-11917929e03a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:36.210562 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae64c0-8a84-481c-b186-11917929e03a service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:36.222947 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1233/2464] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 6 16:23:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:23:36.673418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66c4c9-adae-4fbc-974a-751b2c912625 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:36.678957 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3f66c4c9-adae-4fbc-974a-751b2c912625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:36.804676 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66c4c9-adae-4fbc-974a-751b2c912625 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:36.810434 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99e153-d1e5-4e1d-b57c-4c14bcb4d0b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:36.811238 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99e153-d1e5-4e1d-b57c-4c14bcb4d0b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:36.811623 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99e153-d1e5-4e1d-b57c-4c14bcb4d0b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:36.812001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99e153-d1e5-4e1d-b57c-4c14bcb4d0b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:36.814925 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1a99e153-d1e5-4e1d-b57c-4c14bcb4d0b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:36.916068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99e153-d1e5-4e1d-b57c-4c14bcb4d0b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:36.917588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99e153-d1e5-4e1d-b57c-4c14bcb4d0b6 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:36.935247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:23:36.942282 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1232/2465] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:23:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:23:40.713701 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae64c0-8a84-481c-b186-11917929e03a service swift] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:40.715553 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-50ae64c0-8a84-481c-b186-11917929e03a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:40.733314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae64c0-8a84-481c-b186-11917929e03a service swift] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:40.739781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9e8e6-3ec8-4e95-a510-4a872c472d5a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034d75ab747e4c0cab040e47c6a6236e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:40.740398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9e8e6-3ec8-4e95-a510-4a872c472d5a service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:40.740676 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9e8e6-3ec8-4e95-a510-4a872c472d5a service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:40.740947 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9e8e6-3ec8-4e95-a510-4a872c472d5a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:40.742835 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-acd9e8e6-3ec8-4e95-a510-4a872c472d5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:40.758412 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9e8e6-3ec8-4e95-a510-4a872c472d5a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:40.760091 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9e8e6-3ec8-4e95-a510-4a872c472d5a service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:40.771879 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1234/2466] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:23:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:23:44.365156 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99e153-d1e5-4e1d-b57c-4c14bcb4d0b6 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:44.366896 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1a99e153-d1e5-4e1d-b57c-4c14bcb4d0b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:44.381803 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99e153-d1e5-4e1d-b57c-4c14bcb4d0b6 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:44.388062 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309295b4-ec7c-4d16-895b-1b7b16337bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:44.388744 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309295b4-ec7c-4d16-895b-1b7b16337bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:44.389023 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309295b4-ec7c-4d16-895b-1b7b16337bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:44.389293 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309295b4-ec7c-4d16-895b-1b7b16337bea admin admin] PATH_INFO: `/v3/users/ac26eb671f1a4a889e8d9c7c35a70019` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:44.390616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309295b4-ec7c-4d16-895b-1b7b16337bea admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac26eb671f1a4a889e8d9c7c35a70019)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:44.392803 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309295b4-ec7c-4d16-895b-1b7b16337bea admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:44.456507 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-309295b4-ec7c-4d16-895b-1b7b16337bea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac26eb671f1a4a889e8d9c7c35a70019'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:44.466892 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-309295b4-ec7c-4d16-895b-1b7b16337bea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac26eb671f1a4a889e8d9c7c35a70019'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:44.481116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-309295b4-ec7c-4d16-895b-1b7b16337bea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac26eb671f1a4a889e8d9c7c35a70019'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:44.481892 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-309295b4-ec7c-4d16-895b-1b7b16337bea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac26eb671f1a4a889e8d9c7c35a70019'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:44.482653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-309295b4-ec7c-4d16-895b-1b7b16337bea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac26eb671f1a4a889e8d9c7c35a70019'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:44.496971 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1233/2467] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:23:44 2022] DELETE /identity/v3/users/ac26eb671f1a4a889e8d9c7c35a70019 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:23:44.696461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9e8e6-3ec8-4e95-a510-4a872c472d5a service placement] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:44.713974 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9e8e6-3ec8-4e95-a510-4a872c472d5a service placement] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:44.800537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7f7e8-8ed1-440f-98f1-bce61e467e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:44.801113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7f7e8-8ed1-440f-98f1-bce61e467e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:44.801400 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7f7e8-8ed1-440f-98f1-bce61e467e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:44.801674 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7f7e8-8ed1-440f-98f1-bce61e467e3a admin admin] PATH_INFO: `/v3/projects/fbc78793ec5c47aa9b56c6cbe6c4d9dc` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:44.802876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7f7e8-8ed1-440f-98f1-bce61e467e3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbc78793ec5c47aa9b56c6cbe6c4d9dc)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:44.804800 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7f7e8-8ed1-440f-98f1-bce61e467e3a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:44.858299 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-21e7f7e8-8ed1-440f-98f1-bce61e467e3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbc78793ec5c47aa9b56c6cbe6c4d9dc', 'request_id': 'req-21e7f7e8-8ed1-440f-98f1-bce61e467e3a'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:44.865410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-21e7f7e8-8ed1-440f-98f1-bce61e467e3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbc78793ec5c47aa9b56c6cbe6c4d9dc', 'request_id': 'req-21e7f7e8-8ed1-440f-98f1-bce61e467e3a'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:44.872572 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-21e7f7e8-8ed1-440f-98f1-bce61e467e3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbc78793ec5c47aa9b56c6cbe6c4d9dc', 'request_id': 'req-21e7f7e8-8ed1-440f-98f1-bce61e467e3a'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:44.888315 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-21e7f7e8-8ed1-440f-98f1-bce61e467e3a admin admin] The token cache is being invalidate because project fbc78793ec5c47aa9b56c6cbe6c4d9dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:23:44.888609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-21e7f7e8-8ed1-440f-98f1-bce61e467e3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:44.889434 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-21e7f7e8-8ed1-440f-98f1-bce61e467e3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:44.891391 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1235/2468] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:23:44 2022] DELETE /identity/v3/projects/fbc78793ec5c47aa9b56c6cbe6c4d9dc => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:23:44.895508 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309295b4-ec7c-4d16-895b-1b7b16337bea admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:44.896933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-309295b4-ec7c-4d16-895b-1b7b16337bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:44.916460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309295b4-ec7c-4d16-895b-1b7b16337bea admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:44.986348 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab96a5b-4ab8-4506-8e6f-0599dd184735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:44.987245 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab96a5b-4ab8-4506-8e6f-0599dd184735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:44.987614 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab96a5b-4ab8-4506-8e6f-0599dd184735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:44.987972 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab96a5b-4ab8-4506-8e6f-0599dd184735 admin admin] PATH_INFO: `/v3/users/ad292b1a14684657be07ac57f8ae0ad5` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:44.989382 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab96a5b-4ab8-4506-8e6f-0599dd184735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad292b1a14684657be07ac57f8ae0ad5)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:44.991529 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab96a5b-4ab8-4506-8e6f-0599dd184735 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:45.047553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8ab96a5b-4ab8-4506-8e6f-0599dd184735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad292b1a14684657be07ac57f8ae0ad5'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:45.055033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8ab96a5b-4ab8-4506-8e6f-0599dd184735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad292b1a14684657be07ac57f8ae0ad5'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:45.067439 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:23:45.069072 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8ab96a5b-4ab8-4506-8e6f-0599dd184735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad292b1a14684657be07ac57f8ae0ad5'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:45.069990 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8ab96a5b-4ab8-4506-8e6f-0599dd184735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad292b1a14684657be07ac57f8ae0ad5'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:45.071414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8ab96a5b-4ab8-4506-8e6f-0599dd184735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad292b1a14684657be07ac57f8ae0ad5'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:45.091009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1234/2469] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:23:44 2022] DELETE /identity/v3/users/ad292b1a14684657be07ac57f8ae0ad5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:23:45.309845 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7f7e8-8ed1-440f-98f1-bce61e467e3a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:45.311545 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-21e7f7e8-8ed1-440f-98f1-bce61e467e3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:45.327990 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7f7e8-8ed1-440f-98f1-bce61e467e3a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:45.332013 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:23:45.399533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b5eb3-1eaa-4e04-9cb5-440139b83f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:45.400182 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b5eb3-1eaa-4e04-9cb5-440139b83f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:45.400446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b5eb3-1eaa-4e04-9cb5-440139b83f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:45.400695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b5eb3-1eaa-4e04-9cb5-440139b83f5a admin admin] PATH_INFO: `/v3/projects/613022bf2be14f1aa7f842a6eb63cb35` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:45.401908 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b5eb3-1eaa-4e04-9cb5-440139b83f5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=613022bf2be14f1aa7f842a6eb63cb35)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:45.404377 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b5eb3-1eaa-4e04-9cb5-440139b83f5a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:45.460721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-319b5eb3-1eaa-4e04-9cb5-440139b83f5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '613022bf2be14f1aa7f842a6eb63cb35', 'request_id': 'req-319b5eb3-1eaa-4e04-9cb5-440139b83f5a'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:45.470180 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-319b5eb3-1eaa-4e04-9cb5-440139b83f5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '613022bf2be14f1aa7f842a6eb63cb35', 'request_id': 'req-319b5eb3-1eaa-4e04-9cb5-440139b83f5a'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:45.479618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-319b5eb3-1eaa-4e04-9cb5-440139b83f5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '613022bf2be14f1aa7f842a6eb63cb35', 'request_id': 'req-319b5eb3-1eaa-4e04-9cb5-440139b83f5a'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:45.498685 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-319b5eb3-1eaa-4e04-9cb5-440139b83f5a admin admin] The token cache is being invalidate because project 613022bf2be14f1aa7f842a6eb63cb35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:23:45.499133 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-319b5eb3-1eaa-4e04-9cb5-440139b83f5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:45.499968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-319b5eb3-1eaa-4e04-9cb5-440139b83f5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:23:45.501640 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1236/2470] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:23:45 2022] DELETE /identity/v3/projects/613022bf2be14f1aa7f842a6eb63cb35 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:23:46.918908 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab96a5b-4ab8-4506-8e6f-0599dd184735 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:23:46.920460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8ab96a5b-4ab8-4506-8e6f-0599dd184735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:47.025640 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab96a5b-4ab8-4506-8e6f-0599dd184735 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:23:47.103193 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1e7fc-8a69-41e3-90ae-bf6d42f6c6ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:23:47.103834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1e7fc-8a69-41e3-90ae-bf6d42f6c6ab service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:23:47.104119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1e7fc-8a69-41e3-90ae-bf6d42f6c6ab service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:23:47.104397 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1e7fc-8a69-41e3-90ae-bf6d42f6c6ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:23:47.106897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9da1e7fc-8a69-41e3-90ae-bf6d42f6c6ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:23:47.174352 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1e7fc-8a69-41e3-90ae-bf6d42f6c6ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:23:47.176088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1e7fc-8a69-41e3-90ae-bf6d42f6c6ab service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:23:47.187956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1235/2471] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:23:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:23:55.494366 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:23:57.182486 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=87,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:24:02.290336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5dbc0-64ea-4964-8407-f8dfa6afab24 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:02.291046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5dbc0-64ea-4964-8407-f8dfa6afab24 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:02.291046 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5dbc0-64ea-4964-8407-f8dfa6afab24 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:02.292313 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1237/2472] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 6 16:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:24:02.315197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6055910-774a-4c27-b80e-54f4ad284ac7 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:02.315801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6055910-774a-4c27-b80e-54f4ad284ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:02.315801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6055910-774a-4c27-b80e-54f4ad284ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:02.348220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-c6055910-774a-4c27-b80e-54f4ad284ac7 None None] MFA Rules not processed for user `989beaca146d4f9b950f333bd3567e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:24:02.349621 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c6055910-774a-4c27-b80e-54f4ad284ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:02.403242 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1236/2473] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 6 16:24:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:24:02.415465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5dbc0-64ea-4964-8407-f8dfa6afab24 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:02.417174 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7de5dbc0-64ea-4964-8407-f8dfa6afab24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:02.431241 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5dbc0-64ea-4964-8407-f8dfa6afab24 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:02.435204 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c496096-81d4-4365-8475-554ad1462546 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:02.435824 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c496096-81d4-4365-8475-554ad1462546 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:02.436104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c496096-81d4-4365-8475-554ad1462546 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:02.436378 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c496096-81d4-4365-8475-554ad1462546 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:02.444058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c496096-81d4-4365-8475-554ad1462546 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:02.445487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c496096-81d4-4365-8475-554ad1462546 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:02.453871 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1238/2474] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:24:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:24:02.539842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3aed1-57a4-4611-ae32-7649ad4fb720 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:02.540155 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3aed1-57a4-4611-ae32-7649ad4fb720 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:02.540464 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3aed1-57a4-4611-ae32-7649ad4fb720 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:02.541812 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1237/2475] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 6 16:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:24:02.548163 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646d916-002f-4102-840a-c4b650b054f6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:02.548446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646d916-002f-4102-840a-c4b650b054f6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:02.548719 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646d916-002f-4102-840a-c4b650b054f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:02.573073 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-1646d916-002f-4102-840a-c4b650b054f6 None None] MFA Rules not processed for user `6afd4f6ffdae4b16aa5ba757e68c18c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:24:02.573073 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1646d916-002f-4102-840a-c4b650b054f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:02.580675 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1239/2476] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 6 16:24:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:24:02.591804 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3aed1-57a4-4611-ae32-7649ad4fb720 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:02.603584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3aed1-57a4-4611-ae32-7649ad4fb720 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:02.607847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bdadf6-107e-4692-a2c2-95d43169b9ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:02.608454 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdadf6-107e-4692-a2c2-95d43169b9ca service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:02.608735 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdadf6-107e-4692-a2c2-95d43169b9ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:02.609007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdadf6-107e-4692-a2c2-95d43169b9ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:02.620485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bdadf6-107e-4692-a2c2-95d43169b9ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:02.622338 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bdadf6-107e-4692-a2c2-95d43169b9ca service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:02.633668 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1238/2477] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:24:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:24:05.675556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646d916-002f-4102-840a-c4b650b054f6 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:05.685166 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646d916-002f-4102-840a-c4b650b054f6 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:05.688735 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04123b-2ab7-4304-b010-ad6b03207a63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:05.689326 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04123b-2ab7-4304-b010-ad6b03207a63 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:05.689599 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04123b-2ab7-4304-b010-ad6b03207a63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:05.689864 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04123b-2ab7-4304-b010-ad6b03207a63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:05.691629 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bb04123b-2ab7-4304-b010-ad6b03207a63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:05.762682 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04123b-2ab7-4304-b010-ad6b03207a63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:05.764139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04123b-2ab7-4304-b010-ad6b03207a63 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:05.773964 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1240/2478] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:24:09.226453 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bdadf6-107e-4692-a2c2-95d43169b9ca service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:09.228048 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b5bdadf6-107e-4692-a2c2-95d43169b9ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:09.243566 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bdadf6-107e-4692-a2c2-95d43169b9ca service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:09.249112 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5868590-2165-4c41-ac1f-4de842209d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:09.249832 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5868590-2165-4c41-ac1f-4de842209d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:09.250055 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5868590-2165-4c41-ac1f-4de842209d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:09.250310 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5868590-2165-4c41-ac1f-4de842209d8e admin admin] PATH_INFO: `/v3/users/eda6839efaab4317a0eda1f47f1656a2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:09.265823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5868590-2165-4c41-ac1f-4de842209d8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda6839efaab4317a0eda1f47f1656a2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:09.267823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5868590-2165-4c41-ac1f-4de842209d8e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:09.316999 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f5868590-2165-4c41-ac1f-4de842209d8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda6839efaab4317a0eda1f47f1656a2'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:24:09.325442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f5868590-2165-4c41-ac1f-4de842209d8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda6839efaab4317a0eda1f47f1656a2'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:24:09.337445 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f5868590-2165-4c41-ac1f-4de842209d8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda6839efaab4317a0eda1f47f1656a2'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:24:09.338194 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f5868590-2165-4c41-ac1f-4de842209d8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda6839efaab4317a0eda1f47f1656a2'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:24:09.338842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f5868590-2165-4c41-ac1f-4de842209d8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda6839efaab4317a0eda1f47f1656a2'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:24:09.351733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1239/2479] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:24:09 2022] DELETE /identity/v3/users/eda6839efaab4317a0eda1f47f1656a2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:24:09.523059 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04123b-2ab7-4304-b010-ad6b03207a63 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:09.533072 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04123b-2ab7-4304-b010-ad6b03207a63 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:09.594786 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3acc5a-0d4d-459a-b7cc-b7eda4409e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:09.595423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3acc5a-0d4d-459a-b7cc-b7eda4409e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:09.595702 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3acc5a-0d4d-459a-b7cc-b7eda4409e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:09.595992 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3acc5a-0d4d-459a-b7cc-b7eda4409e62 admin admin] PATH_INFO: `/v3/projects/6d1e05e9e4a940e78a43d397f0c0db16` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:09.608168 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3acc5a-0d4d-459a-b7cc-b7eda4409e62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d1e05e9e4a940e78a43d397f0c0db16)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:09.610137 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3acc5a-0d4d-459a-b7cc-b7eda4409e62 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:09.658239 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7e3acc5a-0d4d-459a-b7cc-b7eda4409e62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d1e05e9e4a940e78a43d397f0c0db16', 'request_id': 'req-7e3acc5a-0d4d-459a-b7cc-b7eda4409e62'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:24:09.664743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7e3acc5a-0d4d-459a-b7cc-b7eda4409e62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d1e05e9e4a940e78a43d397f0c0db16', 'request_id': 'req-7e3acc5a-0d4d-459a-b7cc-b7eda4409e62'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:24:09.671031 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7e3acc5a-0d4d-459a-b7cc-b7eda4409e62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d1e05e9e4a940e78a43d397f0c0db16', 'request_id': 'req-7e3acc5a-0d4d-459a-b7cc-b7eda4409e62'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:24:09.674948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:24:09.687026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7e3acc5a-0d4d-459a-b7cc-b7eda4409e62 admin admin] The token cache is being invalidate because project 6d1e05e9e4a940e78a43d397f0c0db16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:24:09.687319 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7e3acc5a-0d4d-459a-b7cc-b7eda4409e62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:24:09.687994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7e3acc5a-0d4d-459a-b7cc-b7eda4409e62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:24:09.689514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1241/2480] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:24:09 2022] DELETE /identity/v3/projects/6d1e05e9e4a940e78a43d397f0c0db16 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:24:09.695528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501857d7-b70a-471c-8ace-6e8f8759310e None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:09.695806 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501857d7-b70a-471c-8ace-6e8f8759310e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:09.696077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501857d7-b70a-471c-8ace-6e8f8759310e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:09.756021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-501857d7-b70a-471c-8ace-6e8f8759310e None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:24:09.757350 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-501857d7-b70a-471c-8ace-6e8f8759310e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:09.779663 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:24:09.823403 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1240/2481] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:24:09.832530 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a278a9-37d2-48c3-90a9-6e59c4e7d35d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:09.832808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a278a9-37d2-48c3-90a9-6e59c4e7d35d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:09.833082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a278a9-37d2-48c3-90a9-6e59c4e7d35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:09.853372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c5a278a9-37d2-48c3-90a9-6e59c4e7d35d None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:24:09.854615 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c5a278a9-37d2-48c3-90a9-6e59c4e7d35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:09.864418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1242/2482] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:24:09.868556 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501857d7-b70a-471c-8ace-6e8f8759310e None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:09.878308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501857d7-b70a-471c-8ace-6e8f8759310e None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:09.883004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4b1b8-e6b1-475b-95dd-e8152819f735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:09.883600 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4b1b8-e6b1-475b-95dd-e8152819f735 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:09.883880 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4b1b8-e6b1-475b-95dd-e8152819f735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:09.884156 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4b1b8-e6b1-475b-95dd-e8152819f735 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:09.884797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4b1b8-e6b1-475b-95dd-e8152819f735 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:24:09.885097 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4b1b8-e6b1-475b-95dd-e8152819f735 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:09.886747 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4b1b8-e6b1-475b-95dd-e8152819f735 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:09.898259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1241/2483] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:24:09.903595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a278a9-37d2-48c3-90a9-6e59c4e7d35d None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:09.913087 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a278a9-37d2-48c3-90a9-6e59c4e7d35d None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:09.918582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3aeff2-0e7e-42cd-8ae9-5dd45456e3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:09.919169 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3aeff2-0e7e-42cd-8ae9-5dd45456e3db admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:09.919429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3aeff2-0e7e-42cd-8ae9-5dd45456e3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:09.919701 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3aeff2-0e7e-42cd-8ae9-5dd45456e3db admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:09.920364 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3aeff2-0e7e-42cd-8ae9-5dd45456e3db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:09.922275 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3aeff2-0e7e-42cd-8ae9-5dd45456e3db admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:09.960228 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1243/2484] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:24:09 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:24:09.965060 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4b1b8-e6b1-475b-95dd-e8152819f735 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:09.974911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4b1b8-e6b1-475b-95dd-e8152819f735 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:10.036929 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52129250-b8ff-4aa5-9d11-630bfde66ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:10.037540 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52129250-b8ff-4aa5-9d11-630bfde66ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:10.037822 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52129250-b8ff-4aa5-9d11-630bfde66ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:10.038096 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52129250-b8ff-4aa5-9d11-630bfde66ce2 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:10.038770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52129250-b8ff-4aa5-9d11-630bfde66ce2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:10.040701 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52129250-b8ff-4aa5-9d11-630bfde66ce2 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:10.064602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1242/2485] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:24:09 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:24:10.069087 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3aeff2-0e7e-42cd-8ae9-5dd45456e3db admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:10.081146 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3aeff2-0e7e-42cd-8ae9-5dd45456e3db admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:10.086118 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c907aa-6b83-4097-a7af-0dbfa0107063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:10.086837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c907aa-6b83-4097-a7af-0dbfa0107063 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:10.087121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c907aa-6b83-4097-a7af-0dbfa0107063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:10.087422 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c907aa-6b83-4097-a7af-0dbfa0107063 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:10.088123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c907aa-6b83-4097-a7af-0dbfa0107063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:10.090940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c907aa-6b83-4097-a7af-0dbfa0107063 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:10.101740 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1244/2486] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:24:10.107919 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52129250-b8ff-4aa5-9d11-630bfde66ce2 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:10.118027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52129250-b8ff-4aa5-9d11-630bfde66ce2 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:10.122981 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38753397-1a74-434d-a9c6-47ca22bbfe96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:10.123587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38753397-1a74-434d-a9c6-47ca22bbfe96 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:10.123863 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38753397-1a74-434d-a9c6-47ca22bbfe96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:10.124128 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38753397-1a74-434d-a9c6-47ca22bbfe96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:10.125795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38753397-1a74-434d-a9c6-47ca22bbfe96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:10.127473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38753397-1a74-434d-a9c6-47ca22bbfe96 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:10.140965 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1243/2487] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:24:10.145027 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c907aa-6b83-4097-a7af-0dbfa0107063 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:10.156007 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c907aa-6b83-4097-a7af-0dbfa0107063 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:10.160939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae7de9-863f-419d-8bd0-c98d13537dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:10.161598 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae7de9-863f-419d-8bd0-c98d13537dbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:10.161899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae7de9-863f-419d-8bd0-c98d13537dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:10.162184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae7de9-863f-419d-8bd0-c98d13537dbd admin admin] PATH_INFO: `/v3/projects/0679691b1c674dc4968638b8a93d4858/users/66df80c268094c48abf73f1323aa23de/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:10.178326 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae7de9-863f-419d-8bd0-c98d13537dbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0679691b1c674dc4968638b8a93d4858, user_id=66df80c268094c48abf73f1323aa23de, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:10.181026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae7de9-863f-419d-8bd0-c98d13537dbd admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:10.196309 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1245/2488] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:24:10 2022] PUT /identity/v3/projects/0679691b1c674dc4968638b8a93d4858/users/66df80c268094c48abf73f1323aa23de/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:24:10.202598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d06b66-f271-4a9f-ae37-6b60973c2e1d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:10.202914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d06b66-f271-4a9f-ae37-6b60973c2e1d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:10.203190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d06b66-f271-4a9f-ae37-6b60973c2e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:10.223800 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-44d06b66-f271-4a9f-ae37-6b60973c2e1d None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:24:10.225022 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-44d06b66-f271-4a9f-ae37-6b60973c2e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:10.291083 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1244/2489] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:24:10.301646 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae7de9-863f-419d-8bd0-c98d13537dbd admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:10.303402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-61ae7de9-863f-419d-8bd0-c98d13537dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:10.364183 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae7de9-863f-419d-8bd0-c98d13537dbd admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:10.368037 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11207c84-b774-4b77-93eb-df92eeb947e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:10.368651 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11207c84-b774-4b77-93eb-df92eeb947e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:10.368927 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11207c84-b774-4b77-93eb-df92eeb947e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:10.369202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11207c84-b774-4b77-93eb-df92eeb947e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:10.379339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11207c84-b774-4b77-93eb-df92eeb947e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:10.380792 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11207c84-b774-4b77-93eb-df92eeb947e0 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:10.391579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1246/2490] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:24:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:24:14.720671 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482031c1-ace9-4703-97bb-32b747bea21c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:14.721383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482031c1-ace9-4703-97bb-32b747bea21c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:14.721695 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482031c1-ace9-4703-97bb-32b747bea21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:14.761076 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-482031c1-ace9-4703-97bb-32b747bea21c None None] MFA Rules not processed for user `66df80c268094c48abf73f1323aa23de`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:24:14.763319 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-482031c1-ace9-4703-97bb-32b747bea21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:14.789415 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:24:14.860074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1245/2491] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:24:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:24:14.876824 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11207c84-b774-4b77-93eb-df92eeb947e0 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:14.878771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-11207c84-b774-4b77-93eb-df92eeb947e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:14.970795 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11207c84-b774-4b77-93eb-df92eeb947e0 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:14.982275 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70604bed-259a-475c-9c6f-9540c5d3e9e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:14.982884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70604bed-259a-475c-9c6f-9540c5d3e9e4 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:14.983145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70604bed-259a-475c-9c6f-9540c5d3e9e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:14.984302 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70604bed-259a-475c-9c6f-9540c5d3e9e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:14.992994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70604bed-259a-475c-9c6f-9540c5d3e9e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:14.995307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70604bed-259a-475c-9c6f-9540c5d3e9e4 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:15.007524 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1247/2492] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:24:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:24:16.463700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b68cb0-8038-43ba-81db-6083a113a693 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:16.464732 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b68cb0-8038-43ba-81db-6083a113a693 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:16.465768 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b68cb0-8038-43ba-81db-6083a113a693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:16.486912 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-a5b68cb0-8038-43ba-81db-6083a113a693 None None] MFA Rules not processed for user `66df80c268094c48abf73f1323aa23de`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:24:16.488135 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a5b68cb0-8038-43ba-81db-6083a113a693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:16.495012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1246/2493] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:24:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:24:21.888288 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70604bed-259a-475c-9c6f-9540c5d3e9e4 service nova] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:21.897920 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70604bed-259a-475c-9c6f-9540c5d3e9e4 service nova] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:21.901711 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60ee7c-c7ec-40e5-8c2a-141e48a5b202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:21.902324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60ee7c-c7ec-40e5-8c2a-141e48a5b202 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:21.902601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60ee7c-c7ec-40e5-8c2a-141e48a5b202 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:21.902901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60ee7c-c7ec-40e5-8c2a-141e48a5b202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:21.904686 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fa60ee7c-c7ec-40e5-8c2a-141e48a5b202 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:21.919372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60ee7c-c7ec-40e5-8c2a-141e48a5b202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:21.920812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60ee7c-c7ec-40e5-8c2a-141e48a5b202 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:21.925901 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:24:21.936261 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1248/2494] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:24:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:24:23.144569 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b68cb0-8038-43ba-81db-6083a113a693 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:23.146165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a5b68cb0-8038-43ba-81db-6083a113a693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:23.159584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b68cb0-8038-43ba-81db-6083a113a693 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:23.163250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59170c3-03ee-4713-80f6-54d58d5e54e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:23.163891 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59170c3-03ee-4713-80f6-54d58d5e54e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:23.164167 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59170c3-03ee-4713-80f6-54d58d5e54e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:23.164436 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59170c3-03ee-4713-80f6-54d58d5e54e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:23.166163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d59170c3-03ee-4713-80f6-54d58d5e54e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:23.181281 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59170c3-03ee-4713-80f6-54d58d5e54e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:23.182760 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59170c3-03ee-4713-80f6-54d58d5e54e7 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:23.194965 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1247/2495] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:24:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:24:31.937410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:24:33.189798 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=26 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:24:51.347391 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73600b26-5959-462c-aae5-7d9b6379853b None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:51.347962 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73600b26-5959-462c-aae5-7d9b6379853b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:51.347962 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73600b26-5959-462c-aae5-7d9b6379853b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:51.368405 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-73600b26-5959-462c-aae5-7d9b6379853b None None] MFA Rules not processed for user `72cc5f5b1b6b4fb28d57c7ff7931fdff`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:24:51.369618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-73600b26-5959-462c-aae5-7d9b6379853b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:51.425605 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1249/2496] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:24:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:24:51.744751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59170c3-03ee-4713-80f6-54d58d5e54e7 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:51.754576 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59170c3-03ee-4713-80f6-54d58d5e54e7 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:51.758363 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f364d-74dd-4957-92bd-4b1cdae8a953 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:51.759021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f364d-74dd-4957-92bd-4b1cdae8a953 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:51.759280 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f364d-74dd-4957-92bd-4b1cdae8a953 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:51.759550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f364d-74dd-4957-92bd-4b1cdae8a953 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:51.761262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ef6f364d-74dd-4957-92bd-4b1cdae8a953 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:51.772027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f364d-74dd-4957-92bd-4b1cdae8a953 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:51.773463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f364d-74dd-4957-92bd-4b1cdae8a953 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:51.782551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1248/2497] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:24:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:24:52.113967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73600b26-5959-462c-aae5-7d9b6379853b None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:52.115490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-73600b26-5959-462c-aae5-7d9b6379853b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:52.193758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73600b26-5959-462c-aae5-7d9b6379853b None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:52.199107 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f9114-4664-4357-a629-89a2d23c7572 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034d75ab747e4c0cab040e47c6a6236e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:52.199718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f9114-4664-4357-a629-89a2d23c7572 service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:52.199996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f9114-4664-4357-a629-89a2d23c7572 service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:52.200269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f9114-4664-4357-a629-89a2d23c7572 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:52.202002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e79f9114-4664-4357-a629-89a2d23c7572 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:52.213655 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f9114-4664-4357-a629-89a2d23c7572 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:52.215282 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f9114-4664-4357-a629-89a2d23c7572 service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:52.227551 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1250/2498] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:24:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:24:55.459532 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f364d-74dd-4957-92bd-4b1cdae8a953 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:24:55.473742 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f364d-74dd-4957-92bd-4b1cdae8a953 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:24:55.479649 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd322e7-ea54-4d70-b651-ef4b3635746b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:24:55.480340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd322e7-ea54-4d70-b651-ef4b3635746b service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:24:55.480682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd322e7-ea54-4d70-b651-ef4b3635746b service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:24:55.481032 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd322e7-ea54-4d70-b651-ef4b3635746b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:24:55.482903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8fd322e7-ea54-4d70-b651-ef4b3635746b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:24:55.573216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd322e7-ea54-4d70-b651-ef4b3635746b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:24:55.575291 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd322e7-ea54-4d70-b651-ef4b3635746b service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:24:55.600933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1249/2499] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:24:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:02.221830 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=57 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:05.181472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f9114-4664-4357-a629-89a2d23c7572 service placement] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:05.192538 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f9114-4664-4357-a629-89a2d23c7572 service placement] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:05.196167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3a158-315c-4766-a2ea-8fc02dc078ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:05.196767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3a158-315c-4766-a2ea-8fc02dc078ef service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:05.197062 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3a158-315c-4766-a2ea-8fc02dc078ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:05.197313 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3a158-315c-4766-a2ea-8fc02dc078ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:05.199442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c4d3a158-315c-4766-a2ea-8fc02dc078ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:05.210083 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3a158-315c-4766-a2ea-8fc02dc078ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:05.211539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3a158-315c-4766-a2ea-8fc02dc078ef service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:05.221468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1251/2500] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:05.594918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=47 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:11.935121 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd322e7-ea54-4d70-b651-ef4b3635746b service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:11.949610 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd322e7-ea54-4d70-b651-ef4b3635746b service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:11.955780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093ef9c-bafb-4a14-8b7e-6d6c76c4656b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:11.956403 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093ef9c-bafb-4a14-8b7e-6d6c76c4656b service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:11.956683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093ef9c-bafb-4a14-8b7e-6d6c76c4656b service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:11.956961 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093ef9c-bafb-4a14-8b7e-6d6c76c4656b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:11.958822 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2093ef9c-bafb-4a14-8b7e-6d6c76c4656b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:11.977299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2093ef9c-bafb-4a14-8b7e-6d6c76c4656b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:11.979283 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2093ef9c-bafb-4a14-8b7e-6d6c76c4656b service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:11.994019 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1250/2501] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:15.216504 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=6 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:15.581860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2baf1-ef20-4b56-9e72-59736edd34be None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:15.582126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2baf1-ef20-4b56-9e72-59736edd34be None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:15.582617 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2baf1-ef20-4b56-9e72-59736edd34be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:15.612402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-bcd2baf1-ef20-4b56-9e72-59736edd34be None None] MFA Rules not processed for user `95ac6a289fb345d3a98871f5e11f9414`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:25:15.613628 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bcd2baf1-ef20-4b56-9e72-59736edd34be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:15.620960 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1252/2502] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:25:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:25:21.988642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=6 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:25.609428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=6 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:25:27.022880 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093ef9c-bafb-4a14-8b7e-6d6c76c4656b service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:25:27.024814 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2093ef9c-bafb-4a14-8b7e-6d6c76c4656b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:27.038544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093ef9c-bafb-4a14-8b7e-6d6c76c4656b service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:25:27.044095 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03b2bc-3113-416f-81fe-de4348c4c2d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:25:27.044692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03b2bc-3113-416f-81fe-de4348c4c2d9 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:25:27.044965 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03b2bc-3113-416f-81fe-de4348c4c2d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:25:27.045232 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03b2bc-3113-416f-81fe-de4348c4c2d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:25:27.046972 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9f03b2bc-3113-416f-81fe-de4348c4c2d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:25:27.059394 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03b2bc-3113-416f-81fe-de4348c4c2d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:25:27.060997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03b2bc-3113-416f-81fe-de4348c4c2d9 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:25:27.072459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1251/2503] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:25:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:25:37.068532 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=6 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:26:14.512941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2baf1-ef20-4b56-9e72-59736edd34be None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:14.525302 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2baf1-ef20-4b56-9e72-59736edd34be None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:14.529389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309cbb31-ab53-4bb4-a189-7470bf587774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:14.530018 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309cbb31-ab53-4bb4-a189-7470bf587774 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:14.530322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309cbb31-ab53-4bb4-a189-7470bf587774 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:14.530591 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309cbb31-ab53-4bb4-a189-7470bf587774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:14.532540 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-309cbb31-ab53-4bb4-a189-7470bf587774 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:14.546784 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309cbb31-ab53-4bb4-a189-7470bf587774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:14.548268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309cbb31-ab53-4bb4-a189-7470bf587774 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:14.572014 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1253/2504] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:26:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:26:21.874002 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03b2bc-3113-416f-81fe-de4348c4c2d9 service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:21.875569 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9f03b2bc-3113-416f-81fe-de4348c4c2d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:21.890577 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03b2bc-3113-416f-81fe-de4348c4c2d9 service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:21.896669 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda2ed8-1d47-4bcc-8e98-bde08928bb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:21.897310 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda2ed8-1d47-4bcc-8e98-bde08928bb2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:21.897586 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda2ed8-1d47-4bcc-8e98-bde08928bb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:21.897866 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda2ed8-1d47-4bcc-8e98-bde08928bb2f admin admin] PATH_INFO: `/v3/users/e0eaab966616458592a89cdef9cf63c3` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:21.899085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda2ed8-1d47-4bcc-8e98-bde08928bb2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0eaab966616458592a89cdef9cf63c3)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:21.901008 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda2ed8-1d47-4bcc-8e98-bde08928bb2f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:21.955346 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cbda2ed8-1d47-4bcc-8e98-bde08928bb2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0eaab966616458592a89cdef9cf63c3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:21.964181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cbda2ed8-1d47-4bcc-8e98-bde08928bb2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0eaab966616458592a89cdef9cf63c3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:21.976599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cbda2ed8-1d47-4bcc-8e98-bde08928bb2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0eaab966616458592a89cdef9cf63c3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:21.977337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cbda2ed8-1d47-4bcc-8e98-bde08928bb2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0eaab966616458592a89cdef9cf63c3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:21.978010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cbda2ed8-1d47-4bcc-8e98-bde08928bb2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0eaab966616458592a89cdef9cf63c3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:21.992839 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1252/2505] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:26:21 2022] DELETE /identity/v3/users/e0eaab966616458592a89cdef9cf63c3 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:26:22.295720 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309cbb31-ab53-4bb4-a189-7470bf587774 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:22.320649 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309cbb31-ab53-4bb4-a189-7470bf587774 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:22.391189 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe446a-9b92-45b5-aa02-a10f237bbc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:22.391835 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe446a-9b92-45b5-aa02-a10f237bbc46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:22.392120 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe446a-9b92-45b5-aa02-a10f237bbc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:22.392393 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe446a-9b92-45b5-aa02-a10f237bbc46 admin admin] PATH_INFO: `/v3/projects/abf281d8694e486da6d616fe43107474` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:22.393592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe446a-9b92-45b5-aa02-a10f237bbc46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abf281d8694e486da6d616fe43107474)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:22.395792 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe446a-9b92-45b5-aa02-a10f237bbc46 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:22.457740 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bbfe446a-9b92-45b5-aa02-a10f237bbc46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abf281d8694e486da6d616fe43107474', 'request_id': 'req-bbfe446a-9b92-45b5-aa02-a10f237bbc46'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:22.469227 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bbfe446a-9b92-45b5-aa02-a10f237bbc46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abf281d8694e486da6d616fe43107474', 'request_id': 'req-bbfe446a-9b92-45b5-aa02-a10f237bbc46'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:22.481271 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bbfe446a-9b92-45b5-aa02-a10f237bbc46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abf281d8694e486da6d616fe43107474', 'request_id': 'req-bbfe446a-9b92-45b5-aa02-a10f237bbc46'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:22.498397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bbfe446a-9b92-45b5-aa02-a10f237bbc46 admin admin] The token cache is being invalidate because project abf281d8694e486da6d616fe43107474 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:26:22.498726 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bbfe446a-9b92-45b5-aa02-a10f237bbc46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:22.499427 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bbfe446a-9b92-45b5-aa02-a10f237bbc46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:22.500969 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1254/2506] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:26:22 2022] DELETE /identity/v3/projects/abf281d8694e486da6d616fe43107474 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:26:22.506123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda2ed8-1d47-4bcc-8e98-bde08928bb2f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:22.507827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cbda2ed8-1d47-4bcc-8e98-bde08928bb2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:22.523036 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda2ed8-1d47-4bcc-8e98-bde08928bb2f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:22.596600 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71002c7-f159-4f5e-91ff-f7e2a9c065e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:22.597526 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71002c7-f159-4f5e-91ff-f7e2a9c065e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:22.597805 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71002c7-f159-4f5e-91ff-f7e2a9c065e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:22.598061 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71002c7-f159-4f5e-91ff-f7e2a9c065e8 admin admin] PATH_INFO: `/v3/users/20c0e7cd6c6043a39cc712176801493f` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:22.599412 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71002c7-f159-4f5e-91ff-f7e2a9c065e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20c0e7cd6c6043a39cc712176801493f)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:22.602602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71002c7-f159-4f5e-91ff-f7e2a9c065e8 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:22.666684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e71002c7-f159-4f5e-91ff-f7e2a9c065e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20c0e7cd6c6043a39cc712176801493f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:22.677380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e71002c7-f159-4f5e-91ff-f7e2a9c065e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20c0e7cd6c6043a39cc712176801493f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:22.695815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e71002c7-f159-4f5e-91ff-f7e2a9c065e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20c0e7cd6c6043a39cc712176801493f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:22.696842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e71002c7-f159-4f5e-91ff-f7e2a9c065e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20c0e7cd6c6043a39cc712176801493f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:22.697708 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e71002c7-f159-4f5e-91ff-f7e2a9c065e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20c0e7cd6c6043a39cc712176801493f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:22.701172 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:26:22.715258 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1253/2507] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:26:22 2022] DELETE /identity/v3/users/20c0e7cd6c6043a39cc712176801493f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:26:22.920123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe446a-9b92-45b5-aa02-a10f237bbc46 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:22.921808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bbfe446a-9b92-45b5-aa02-a10f237bbc46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:22.937937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe446a-9b92-45b5-aa02-a10f237bbc46 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:23.027780 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa524b-34f2-48bb-8c49-26b090ddab78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:23.028328 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa524b-34f2-48bb-8c49-26b090ddab78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:23.028604 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa524b-34f2-48bb-8c49-26b090ddab78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:23.028875 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa524b-34f2-48bb-8c49-26b090ddab78 admin admin] PATH_INFO: `/v3/projects/6b00e1d8e21d4306b25c7361c3f9e305` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:23.030016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa524b-34f2-48bb-8c49-26b090ddab78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b00e1d8e21d4306b25c7361c3f9e305)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:23.031972 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa524b-34f2-48bb-8c49-26b090ddab78 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:23.054614 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:26:23.084607 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-27aa524b-34f2-48bb-8c49-26b090ddab78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b00e1d8e21d4306b25c7361c3f9e305', 'request_id': 'req-27aa524b-34f2-48bb-8c49-26b090ddab78'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:23.091276 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-27aa524b-34f2-48bb-8c49-26b090ddab78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b00e1d8e21d4306b25c7361c3f9e305', 'request_id': 'req-27aa524b-34f2-48bb-8c49-26b090ddab78'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:23.099463 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-27aa524b-34f2-48bb-8c49-26b090ddab78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b00e1d8e21d4306b25c7361c3f9e305', 'request_id': 'req-27aa524b-34f2-48bb-8c49-26b090ddab78'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:23.112088 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-27aa524b-34f2-48bb-8c49-26b090ddab78 admin admin] The token cache is being invalidate because project 6b00e1d8e21d4306b25c7361c3f9e305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:26:23.112374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-27aa524b-34f2-48bb-8c49-26b090ddab78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:23.113041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-27aa524b-34f2-48bb-8c49-26b090ddab78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:26:23.114715 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1255/2508] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:26:22 2022] DELETE /identity/v3/projects/6b00e1d8e21d4306b25c7361c3f9e305 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:26:23.119882 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95165c7-f210-4808-8c3d-0a72c6e974d2 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:23.120415 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95165c7-f210-4808-8c3d-0a72c6e974d2 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:23.120415 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95165c7-f210-4808-8c3d-0a72c6e974d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:23.143645 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-c95165c7-f210-4808-8c3d-0a72c6e974d2 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:26:23.145560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c95165c7-f210-4808-8c3d-0a72c6e974d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:23.208278 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1254/2509] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:26:23.216513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ef951-6dfb-4f64-aaca-450d1e21a47e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:23.216797 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ef951-6dfb-4f64-aaca-450d1e21a47e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:23.217070 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ef951-6dfb-4f64-aaca-450d1e21a47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:23.237324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-bd3ef951-6dfb-4f64-aaca-450d1e21a47e None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:26:23.238926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bd3ef951-6dfb-4f64-aaca-450d1e21a47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:23.247907 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1256/2510] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:26:23.252427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95165c7-f210-4808-8c3d-0a72c6e974d2 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:23.266408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95165c7-f210-4808-8c3d-0a72c6e974d2 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:23.274492 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c737a8-6312-4026-b2bb-9dd42f513cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:23.275174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c737a8-6312-4026-b2bb-9dd42f513cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:23.275452 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c737a8-6312-4026-b2bb-9dd42f513cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:23.275722 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c737a8-6312-4026-b2bb-9dd42f513cf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:23.276380 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c737a8-6312-4026-b2bb-9dd42f513cf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:26:23.276682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c737a8-6312-4026-b2bb-9dd42f513cf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:23.278336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c737a8-6312-4026-b2bb-9dd42f513cf9 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:23.291193 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1255/2511] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:26:23.295743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ef951-6dfb-4f64-aaca-450d1e21a47e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:23.305771 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ef951-6dfb-4f64-aaca-450d1e21a47e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:23.310623 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfc2a9-cf70-44da-9ecf-676a3cd07008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:23.311302 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfc2a9-cf70-44da-9ecf-676a3cd07008 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:23.311572 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfc2a9-cf70-44da-9ecf-676a3cd07008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:23.311835 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfc2a9-cf70-44da-9ecf-676a3cd07008 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:23.312646 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfc2a9-cf70-44da-9ecf-676a3cd07008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:23.314502 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfc2a9-cf70-44da-9ecf-676a3cd07008 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:23.355665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1257/2512] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:26:23 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:26:23.357589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c737a8-6312-4026-b2bb-9dd42f513cf9 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:23.370020 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c737a8-6312-4026-b2bb-9dd42f513cf9 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:23.432409 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6c0b5-0d3b-4726-b9e4-aecd8cfccc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:23.433056 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6c0b5-0d3b-4726-b9e4-aecd8cfccc45 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:23.433336 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6c0b5-0d3b-4726-b9e4-aecd8cfccc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:23.433609 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6c0b5-0d3b-4726-b9e4-aecd8cfccc45 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:23.434253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6c0b5-0d3b-4726-b9e4-aecd8cfccc45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:23.436220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6c0b5-0d3b-4726-b9e4-aecd8cfccc45 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:23.460824 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1256/2513] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:26:23 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:26:23.464359 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfc2a9-cf70-44da-9ecf-676a3cd07008 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:23.480042 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfc2a9-cf70-44da-9ecf-676a3cd07008 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:23.485780 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3053f6-9ccd-4036-acfa-8b43827a7c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:23.486325 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3053f6-9ccd-4036-acfa-8b43827a7c58 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:23.486617 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3053f6-9ccd-4036-acfa-8b43827a7c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:23.486945 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3053f6-9ccd-4036-acfa-8b43827a7c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:23.491464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3053f6-9ccd-4036-acfa-8b43827a7c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:23.493086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3053f6-9ccd-4036-acfa-8b43827a7c58 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:23.503657 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1258/2514] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:26:23.508686 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6c0b5-0d3b-4726-b9e4-aecd8cfccc45 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:23.519206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6c0b5-0d3b-4726-b9e4-aecd8cfccc45 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:23.524761 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7033c19-2c21-4b34-877f-d4ed6382c40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:23.525366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7033c19-2c21-4b34-877f-d4ed6382c40f admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:23.525670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7033c19-2c21-4b34-877f-d4ed6382c40f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:23.525906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7033c19-2c21-4b34-877f-d4ed6382c40f admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:23.526547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7033c19-2c21-4b34-877f-d4ed6382c40f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:23.528199 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7033c19-2c21-4b34-877f-d4ed6382c40f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:23.540691 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1257/2515] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:26:23.544930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3053f6-9ccd-4036-acfa-8b43827a7c58 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:23.555545 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3053f6-9ccd-4036-acfa-8b43827a7c58 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:23.560287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098661c-1b57-4ece-ae58-c3d097c5bb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:23.560948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098661c-1b57-4ece-ae58-c3d097c5bb50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:23.561242 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098661c-1b57-4ece-ae58-c3d097c5bb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:23.561509 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098661c-1b57-4ece-ae58-c3d097c5bb50 admin admin] PATH_INFO: `/v3/projects/870b3a5a8fc74957bf004f0a6f6020a3/users/210d0f461d2c445a9688bc233d048079/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:23.577663 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098661c-1b57-4ece-ae58-c3d097c5bb50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=870b3a5a8fc74957bf004f0a6f6020a3, user_id=210d0f461d2c445a9688bc233d048079, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:23.580390 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098661c-1b57-4ece-ae58-c3d097c5bb50 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:23.594783 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1259/2516] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:26:23 2022] PUT /identity/v3/projects/870b3a5a8fc74957bf004f0a6f6020a3/users/210d0f461d2c445a9688bc233d048079/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:26:23.599468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4348b-55b4-490b-b3b1-9d59dd94ba92 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:23.599869 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4348b-55b4-490b-b3b1-9d59dd94ba92 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:23.600211 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4348b-55b4-490b-b3b1-9d59dd94ba92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:23.621385 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f4d4348b-55b4-490b-b3b1-9d59dd94ba92 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:26:23.622620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f4d4348b-55b4-490b-b3b1-9d59dd94ba92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:23.676436 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:26:23.687952 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1258/2517] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:26:23.698216 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098661c-1b57-4ece-ae58-c3d097c5bb50 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:23.699974 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c098661c-1b57-4ece-ae58-c3d097c5bb50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:23.779228 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098661c-1b57-4ece-ae58-c3d097c5bb50 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:23.785846 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f77abf-e19a-4b53-a2d4-e7390f148902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:23.786631 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f77abf-e19a-4b53-a2d4-e7390f148902 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:23.786949 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f77abf-e19a-4b53-a2d4-e7390f148902 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:23.787224 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f77abf-e19a-4b53-a2d4-e7390f148902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:23.797704 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f77abf-e19a-4b53-a2d4-e7390f148902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:23.799130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f77abf-e19a-4b53-a2d4-e7390f148902 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:23.810587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1260/2518] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:26:29.298814 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a60d76-cf1a-4d03-9731-2376b2556af0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:29.299400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a60d76-cf1a-4d03-9731-2376b2556af0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:29.299400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a60d76-cf1a-4d03-9731-2376b2556af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:29.333135 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-c0a60d76-cf1a-4d03-9731-2376b2556af0 None None] MFA Rules not processed for user `210d0f461d2c445a9688bc233d048079`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:26:29.335233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c0a60d76-cf1a-4d03-9731-2376b2556af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:29.425323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1259/2519] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:26:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:26:29.431584 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f77abf-e19a-4b53-a2d4-e7390f148902 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:29.445078 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f77abf-e19a-4b53-a2d4-e7390f148902 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:29.450225 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82316a-bcd8-4ecc-b5b5-ca8c68b31ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:29.451807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82316a-bcd8-4ecc-b5b5-ca8c68b31ecf admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:29.451807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82316a-bcd8-4ecc-b5b5-ca8c68b31ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:29.452049 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82316a-bcd8-4ecc-b5b5-ca8c68b31ecf admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:29.452672 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc82316a-bcd8-4ecc-b5b5-ca8c68b31ecf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:29.455092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc82316a-bcd8-4ecc-b5b5-ca8c68b31ecf admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:29.488955 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1261/2520] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:26:29 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:26:29.493753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a60d76-cf1a-4d03-9731-2376b2556af0 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:29.505758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a60d76-cf1a-4d03-9731-2376b2556af0 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:29.579106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a3003-aa95-4666-892b-3dc1237f0388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:29.579707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a3003-aa95-4666-892b-3dc1237f0388 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:29.580114 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a3003-aa95-4666-892b-3dc1237f0388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:29.580389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a3003-aa95-4666-892b-3dc1237f0388 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:29.581044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a3003-aa95-4666-892b-3dc1237f0388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:29.583946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a3003-aa95-4666-892b-3dc1237f0388 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:29.612926 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1260/2521] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:26:29 2022] POST /identity/v3/users => generated 296 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:26:29.618162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82316a-bcd8-4ecc-b5b5-ca8c68b31ecf admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:29.629839 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82316a-bcd8-4ecc-b5b5-ca8c68b31ecf admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:29.634904 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d409b-eb81-4275-9554-ff7064d028a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:29.635553 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d409b-eb81-4275-9554-ff7064d028a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:29.635863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d409b-eb81-4275-9554-ff7064d028a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:29.636130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d409b-eb81-4275-9554-ff7064d028a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:29.636776 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d409b-eb81-4275-9554-ff7064d028a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:29.638414 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d409b-eb81-4275-9554-ff7064d028a9 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:29.651340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1262/2522] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:26:29.655664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a3003-aa95-4666-892b-3dc1237f0388 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:29.668850 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a3003-aa95-4666-892b-3dc1237f0388 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:29.674732 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86c0c7-33ce-4d5c-b05e-5c5f945544ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:29.676043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86c0c7-33ce-4d5c-b05e-5c5f945544ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:29.676438 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86c0c7-33ce-4d5c-b05e-5c5f945544ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:29.676735 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86c0c7-33ce-4d5c-b05e-5c5f945544ff admin admin] PATH_INFO: `/v3/projects/082bb335eea440d0a1145339ee9610aa/users/fd9ff285c2c8473faa96355a927bc4e7/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:29.702879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86c0c7-33ce-4d5c-b05e-5c5f945544ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=082bb335eea440d0a1145339ee9610aa, user_id=fd9ff285c2c8473faa96355a927bc4e7, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:29.708635 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86c0c7-33ce-4d5c-b05e-5c5f945544ff admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:29.730134 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1261/2523] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:26:29 2022] PUT /identity/v3/projects/082bb335eea440d0a1145339ee9610aa/users/fd9ff285c2c8473faa96355a927bc4e7/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:26:29.734232 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d409b-eb81-4275-9554-ff7064d028a9 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:29.745686 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d409b-eb81-4275-9554-ff7064d028a9 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:29.757051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:26:29.818627 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72493cba-2802-4675-baa3-d2f7b7c5dd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:29.819380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72493cba-2802-4675-baa3-d2f7b7c5dd14 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:29.819660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72493cba-2802-4675-baa3-d2f7b7c5dd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:29.819929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72493cba-2802-4675-baa3-d2f7b7c5dd14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:29.820585 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72493cba-2802-4675-baa3-d2f7b7c5dd14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:29.822221 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72493cba-2802-4675-baa3-d2f7b7c5dd14 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:29.834412 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1263/2524] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:26:29.839202 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86c0c7-33ce-4d5c-b05e-5c5f945544ff admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:29.851296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86c0c7-33ce-4d5c-b05e-5c5f945544ff admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:29.857098 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f3ac2-8dd3-4b4b-bf98-db34d1944ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:29.857771 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f3ac2-8dd3-4b4b-bf98-db34d1944ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:29.858049 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f3ac2-8dd3-4b4b-bf98-db34d1944ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:29.858323 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f3ac2-8dd3-4b4b-bf98-db34d1944ac5 admin admin] PATH_INFO: `/v3/projects/082bb335eea440d0a1145339ee9610aa/users/fd9ff285c2c8473faa96355a927bc4e7/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:29.860917 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f3ac2-8dd3-4b4b-bf98-db34d1944ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=082bb335eea440d0a1145339ee9610aa, user_id=fd9ff285c2c8473faa96355a927bc4e7, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:29.865549 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f3ac2-8dd3-4b4b-bf98-db34d1944ac5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:29.879461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1262/2525] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:26:29 2022] PUT /identity/v3/projects/082bb335eea440d0a1145339ee9610aa/users/fd9ff285c2c8473faa96355a927bc4e7/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:26:35.067732 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67e661-bfe0-4082-b094-d754f213db6e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:35.068560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67e661-bfe0-4082-b094-d754f213db6e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:35.068560 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67e661-bfe0-4082-b094-d754f213db6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:35.117811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-3c67e661-bfe0-4082-b094-d754f213db6e None None] MFA Rules not processed for user `fd9ff285c2c8473faa96355a927bc4e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:26:35.120240 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3c67e661-bfe0-4082-b094-d754f213db6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:35.176189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f3ac2-8dd3-4b4b-bf98-db34d1944ac5 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:35.178341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-352f3ac2-8dd3-4b4b-bf98-db34d1944ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:35.256865 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:26:35.260044 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1264/2526] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:26:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:26:35.388069 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f3ac2-8dd3-4b4b-bf98-db34d1944ac5 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:35.401535 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f68c26-d331-424e-ba88-dc643e395f81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:35.404765 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f68c26-d331-424e-ba88-dc643e395f81 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:35.405872 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f68c26-d331-424e-ba88-dc643e395f81 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:35.406788 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f68c26-d331-424e-ba88-dc643e395f81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:35.411799 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-93f68c26-d331-424e-ba88-dc643e395f81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:35.465022 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67e661-bfe0-4082-b094-d754f213db6e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:35.467201 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3c67e661-bfe0-4082-b094-d754f213db6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:35.489402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67e661-bfe0-4082-b094-d754f213db6e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:35.498731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886284f-7f9c-4f32-a013-7b5d5b884f4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:35.498731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886284f-7f9c-4f32-a013-7b5d5b884f4d service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:35.498731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886284f-7f9c-4f32-a013-7b5d5b884f4d service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:35.498731 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886284f-7f9c-4f32-a013-7b5d5b884f4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:35.611972 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886284f-7f9c-4f32-a013-7b5d5b884f4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:35.612201 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f68c26-d331-424e-ba88-dc643e395f81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:35.614624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f68c26-d331-424e-ba88-dc643e395f81 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:35.617103 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886284f-7f9c-4f32-a013-7b5d5b884f4d service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:35.639827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1265/2527] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:26:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:26:35.646911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1263/2528] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:26:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:26:39.682132 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3dd75-d9ee-4a61-b3d2-b82f5b7dfbfe None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:39.682749 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3dd75-d9ee-4a61-b3d2-b82f5b7dfbfe None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:39.682853 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3dd75-d9ee-4a61-b3d2-b82f5b7dfbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:39.772290 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-14b3dd75-d9ee-4a61-b3d2-b82f5b7dfbfe None None] MFA Rules not processed for user `ba58a4914ddf418db9e8b9dce5784cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:26:39.775086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-14b3dd75-d9ee-4a61-b3d2-b82f5b7dfbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:39.784084 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:26:39.905474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1266/2529] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 6 16:26:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:26:40.015361 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f68c26-d331-424e-ba88-dc643e395f81 service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:26:40.017978 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-93f68c26-d331-424e-ba88-dc643e395f81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:26:40.150946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f68c26-d331-424e-ba88-dc643e395f81 service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:26:40.156433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0d03d-8b0a-4c78-abe0-c520ec59b44e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ece176d412574790a13d03dcca08f91a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:26:40.157115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0d03d-8b0a-4c78-abe0-c520ec59b44e service swift] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:26:40.157397 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0d03d-8b0a-4c78-abe0-c520ec59b44e service swift] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:26:40.157661 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0d03d-8b0a-4c78-abe0-c520ec59b44e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:26:40.167649 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0d03d-8b0a-4c78-abe0-c520ec59b44e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:26:40.169305 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0d03d-8b0a-4c78-abe0-c520ec59b44e service swift] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:26:40.180669 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1264/2530] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 6 16:26:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:26:49.917688 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=25 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:26:50.174714 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:03.032319 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a2621-0fce-4aae-b770-e08db69e580c None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:03.033460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a2621-0fce-4aae-b770-e08db69e580c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:03.034010 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a2621-0fce-4aae-b770-e08db69e580c None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:03.037315 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1267/2531] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 6 16:27:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 16:27:03.049316 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db8f60-04a3-4ef7-a078-8257cb463246 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:03.049821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db8f60-04a3-4ef7-a078-8257cb463246 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:03.049941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db8f60-04a3-4ef7-a078-8257cb463246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:03.116985 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-96db8f60-04a3-4ef7-a078-8257cb463246 None None] MFA Rules not processed for user `6afd4f6ffdae4b16aa5ba757e68c18c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:03.118349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-96db8f60-04a3-4ef7-a078-8257cb463246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:03.133760 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1265/2532] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 6 16:27:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:03.154391 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a2621-0fce-4aae-b770-e08db69e580c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:03.157359 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1c0a2621-0fce-4aae-b770-e08db69e580c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:03.282372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a2621-0fce-4aae-b770-e08db69e580c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:03.291869 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec4c33-4d52-455d-af5e-cde44c8bcbab service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50ef7db27744c7ebd2ec95cecc9d1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:03.292542 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec4c33-4d52-455d-af5e-cde44c8bcbab service glance] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:03.292843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec4c33-4d52-455d-af5e-cde44c8bcbab service glance] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:03.293096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec4c33-4d52-455d-af5e-cde44c8bcbab service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:03.306342 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec4c33-4d52-455d-af5e-cde44c8bcbab service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:03.307867 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec4c33-4d52-455d-af5e-cde44c8bcbab service glance] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:03.338678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1268/2533] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:04.051999 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7d68e-7a83-4c2d-ac75-abd889757b75 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:04.052737 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7d68e-7a83-4c2d-ac75-abd889757b75 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:04.052737 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7d68e-7a83-4c2d-ac75-abd889757b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:04.083315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1ec7d68e-7a83-4c2d-ac75-abd889757b75 None None] MFA Rules not processed for user `210d0f461d2c445a9688bc233d048079`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:04.085666 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1ec7d68e-7a83-4c2d-ac75-abd889757b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:04.096982 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1266/2534] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:04.620863 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec4c33-4d52-455d-af5e-cde44c8bcbab service glance] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:04.622610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0aec4c33-4d52-455d-af5e-cde44c8bcbab service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:04.719776 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec4c33-4d52-455d-af5e-cde44c8bcbab service glance] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:04.724454 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aee512-d3ea-4b51-b1f3-e49e0d94732d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:04.725093 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aee512-d3ea-4b51-b1f3-e49e0d94732d service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:04.725369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aee512-d3ea-4b51-b1f3-e49e0d94732d service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:04.725638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aee512-d3ea-4b51-b1f3-e49e0d94732d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:04.727703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b0aee512-d3ea-4b51-b1f3-e49e0d94732d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:04.739280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aee512-d3ea-4b51-b1f3-e49e0d94732d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:04.740745 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aee512-d3ea-4b51-b1f3-e49e0d94732d service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:04.749654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1269/2535] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:27:09.739607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322b1c5-856c-444f-917f-327382483ca6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:09.740213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322b1c5-856c-444f-917f-327382483ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:09.740717 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322b1c5-856c-444f-917f-327382483ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:09.774526 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-7322b1c5-856c-444f-917f-327382483ca6 None None] MFA Rules not processed for user `ba58a4914ddf418db9e8b9dce5784cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:09.776586 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7322b1c5-856c-444f-917f-327382483ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:09.785934 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1267/2536] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 6 16:27:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:09.802034 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aee512-d3ea-4b51-b1f3-e49e0d94732d service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:09.829000 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aee512-d3ea-4b51-b1f3-e49e0d94732d service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:09.839519 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f1ef7-800c-47f8-941b-d00cfadd10e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ece176d412574790a13d03dcca08f91a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:09.842193 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f1ef7-800c-47f8-941b-d00cfadd10e1 service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:09.843127 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f1ef7-800c-47f8-941b-d00cfadd10e1 service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:09.844299 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f1ef7-800c-47f8-941b-d00cfadd10e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:09.861194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f1ef7-800c-47f8-941b-d00cfadd10e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:09.863596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f1ef7-800c-47f8-941b-d00cfadd10e1 service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:09.874900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1270/2537] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 6 16:27:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:27:19.770400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=15 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:19.870653 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=67 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:30.360490 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322b1c5-856c-444f-917f-327382483ca6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:30.378131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322b1c5-856c-444f-917f-327382483ca6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:30.382541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb75e5-cc61-43d2-b884-1479e3ced4f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:30.383265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb75e5-cc61-43d2-b884-1479e3ced4f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:30.383549 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb75e5-cc61-43d2-b884-1479e3ced4f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:30.383834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb75e5-cc61-43d2-b884-1479e3ced4f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:30.385762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d5cb75e5-cc61-43d2-b884-1479e3ced4f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:30.479659 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb75e5-cc61-43d2-b884-1479e3ced4f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:30.481190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb75e5-cc61-43d2-b884-1479e3ced4f8 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:30.494359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1268/2538] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:33.994405 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f1ef7-800c-47f8-941b-d00cfadd10e1 service swift] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:33.996994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b68f1ef7-800c-47f8-941b-d00cfadd10e1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:34.088273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f1ef7-800c-47f8-941b-d00cfadd10e1 service swift] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:34.097144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2829db7-6469-4ca8-be06-6e138b79107a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034d75ab747e4c0cab040e47c6a6236e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:34.098896 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2829db7-6469-4ca8-be06-6e138b79107a service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:34.099238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2829db7-6469-4ca8-be06-6e138b79107a service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:34.099503 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2829db7-6469-4ca8-be06-6e138b79107a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:34.103305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c2829db7-6469-4ca8-be06-6e138b79107a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:34.128022 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2829db7-6469-4ca8-be06-6e138b79107a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:34.129959 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2829db7-6469-4ca8-be06-6e138b79107a service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:34.130377 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb75e5-cc61-43d2-b884-1479e3ced4f8 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:34.132317 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d5cb75e5-cc61-43d2-b884-1479e3ced4f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:34.144479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1271/2539] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:34.150265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb75e5-cc61-43d2-b884-1479e3ced4f8 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:34.156535 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e80a00-688a-43df-b289-95f8f61d379e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:34.157750 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e80a00-688a-43df-b289-95f8f61d379e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:34.158196 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e80a00-688a-43df-b289-95f8f61d379e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:34.158752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e80a00-688a-43df-b289-95f8f61d379e admin admin] PATH_INFO: `/v3/users/72cc5f5b1b6b4fb28d57c7ff7931fdff` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:34.160395 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e80a00-688a-43df-b289-95f8f61d379e admin admin] RBAC: Authorizing `identity:delete_user(user_id=72cc5f5b1b6b4fb28d57c7ff7931fdff)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:34.163565 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e80a00-688a-43df-b289-95f8f61d379e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:34.262634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-34e80a00-688a-43df-b289-95f8f61d379e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72cc5f5b1b6b4fb28d57c7ff7931fdff'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:34.274958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-34e80a00-688a-43df-b289-95f8f61d379e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72cc5f5b1b6b4fb28d57c7ff7931fdff'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:34.299237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-34e80a00-688a-43df-b289-95f8f61d379e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72cc5f5b1b6b4fb28d57c7ff7931fdff'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:34.300093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-34e80a00-688a-43df-b289-95f8f61d379e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72cc5f5b1b6b4fb28d57c7ff7931fdff'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:34.300713 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-34e80a00-688a-43df-b289-95f8f61d379e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72cc5f5b1b6b4fb28d57c7ff7931fdff'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:34.320663 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1269/2540] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:34 2022] DELETE /identity/v3/users/72cc5f5b1b6b4fb28d57c7ff7931fdff => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:27:34.524555 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2829db7-6469-4ca8-be06-6e138b79107a service placement] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:34.543320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2829db7-6469-4ca8-be06-6e138b79107a service placement] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:34.633076 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc020da-e3d1-4af4-8acc-4fa06d5f1b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:34.633857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc020da-e3d1-4af4-8acc-4fa06d5f1b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:34.634206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc020da-e3d1-4af4-8acc-4fa06d5f1b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:34.634591 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc020da-e3d1-4af4-8acc-4fa06d5f1b66 admin admin] PATH_INFO: `/v3/projects/b52ac4a27fe84303970a9f7e104c5298` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:34.636086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc020da-e3d1-4af4-8acc-4fa06d5f1b66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b52ac4a27fe84303970a9f7e104c5298)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:34.638608 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc020da-e3d1-4af4-8acc-4fa06d5f1b66 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:34.700828 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3cc020da-e3d1-4af4-8acc-4fa06d5f1b66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b52ac4a27fe84303970a9f7e104c5298', 'request_id': 'req-3cc020da-e3d1-4af4-8acc-4fa06d5f1b66'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:34.708809 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3cc020da-e3d1-4af4-8acc-4fa06d5f1b66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b52ac4a27fe84303970a9f7e104c5298', 'request_id': 'req-3cc020da-e3d1-4af4-8acc-4fa06d5f1b66'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:34.717755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3cc020da-e3d1-4af4-8acc-4fa06d5f1b66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b52ac4a27fe84303970a9f7e104c5298', 'request_id': 'req-3cc020da-e3d1-4af4-8acc-4fa06d5f1b66'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:34.733927 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3cc020da-e3d1-4af4-8acc-4fa06d5f1b66 admin admin] The token cache is being invalidate because project b52ac4a27fe84303970a9f7e104c5298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:34.734270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3cc020da-e3d1-4af4-8acc-4fa06d5f1b66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:34.734926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3cc020da-e3d1-4af4-8acc-4fa06d5f1b66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:34.736399 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1272/2541] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:34 2022] DELETE /identity/v3/projects/b52ac4a27fe84303970a9f7e104c5298 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:34.742736 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0eaec-a561-438f-8022-a00c1b58994a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:34.743249 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0eaec-a561-438f-8022-a00c1b58994a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:34.743709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0eaec-a561-438f-8022-a00c1b58994a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:34.769605 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1ed0eaec-a561-438f-8022-a00c1b58994a None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:34.771408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1ed0eaec-a561-438f-8022-a00c1b58994a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:34.838425 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1270/2542] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:34.846248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea96b9-5865-46cd-b76b-bbe524312f88 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:34.846610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea96b9-5865-46cd-b76b-bbe524312f88 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:34.846982 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea96b9-5865-46cd-b76b-bbe524312f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:34.871849 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6dea96b9-5865-46cd-b76b-bbe524312f88 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:34.873075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6dea96b9-5865-46cd-b76b-bbe524312f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:34.882044 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1273/2543] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:34.886444 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0eaec-a561-438f-8022-a00c1b58994a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:34.900637 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0eaec-a561-438f-8022-a00c1b58994a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:34.905956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a0b93-d5b8-4202-b369-8a9cef03eb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:34.906691 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a0b93-d5b8-4202-b369-8a9cef03eb7b admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:34.907207 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a0b93-d5b8-4202-b369-8a9cef03eb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:34.907553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a0b93-d5b8-4202-b369-8a9cef03eb7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:34.908328 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a0b93-d5b8-4202-b369-8a9cef03eb7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:27:34.908734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a0b93-d5b8-4202-b369-8a9cef03eb7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:34.911208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a0b93-d5b8-4202-b369-8a9cef03eb7b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:34.912959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:34.935081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1271/2544] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:27:34.936204 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea96b9-5865-46cd-b76b-bbe524312f88 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:34.948330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea96b9-5865-46cd-b76b-bbe524312f88 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:34.953439 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b86ba-5456-4caa-84c4-5a318b2b0143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:34.954058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b86ba-5456-4caa-84c4-5a318b2b0143 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:34.954338 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b86ba-5456-4caa-84c4-5a318b2b0143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:34.954609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b86ba-5456-4caa-84c4-5a318b2b0143 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:34.955285 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b86ba-5456-4caa-84c4-5a318b2b0143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:34.958232 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b86ba-5456-4caa-84c4-5a318b2b0143 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:34.970148 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:34.990720 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1274/2545] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:34 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:34.995021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a0b93-d5b8-4202-b369-8a9cef03eb7b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:35.005754 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a0b93-d5b8-4202-b369-8a9cef03eb7b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:35.067384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31e6cb-91c1-4671-b99c-24b5c68d596b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:35.068045 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31e6cb-91c1-4671-b99c-24b5c68d596b admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:35.068319 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31e6cb-91c1-4671-b99c-24b5c68d596b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:35.068587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31e6cb-91c1-4671-b99c-24b5c68d596b admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:35.069215 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31e6cb-91c1-4671-b99c-24b5c68d596b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:35.071151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31e6cb-91c1-4671-b99c-24b5c68d596b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:35.094108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1272/2546] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:34 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:27:35.098554 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b86ba-5456-4caa-84c4-5a318b2b0143 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:35.109997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b86ba-5456-4caa-84c4-5a318b2b0143 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:35.114736 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544342ae-fb70-46fe-8ca4-c918abe96a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:35.115413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544342ae-fb70-46fe-8ca4-c918abe96a60 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:35.115613 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544342ae-fb70-46fe-8ca4-c918abe96a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:35.115882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544342ae-fb70-46fe-8ca4-c918abe96a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:35.116531 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544342ae-fb70-46fe-8ca4-c918abe96a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:35.118113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544342ae-fb70-46fe-8ca4-c918abe96a60 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:35.129574 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1275/2547] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:35.133581 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31e6cb-91c1-4671-b99c-24b5c68d596b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:35.144523 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31e6cb-91c1-4671-b99c-24b5c68d596b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:35.150043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796bc80-c7f7-47de-840c-fc370f77454c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:35.150646 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796bc80-c7f7-47de-840c-fc370f77454c admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:35.150947 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796bc80-c7f7-47de-840c-fc370f77454c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:35.151220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796bc80-c7f7-47de-840c-fc370f77454c admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:35.151851 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796bc80-c7f7-47de-840c-fc370f77454c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:35.153441 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796bc80-c7f7-47de-840c-fc370f77454c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:35.164686 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1273/2548] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:35.168929 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544342ae-fb70-46fe-8ca4-c918abe96a60 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:35.179302 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544342ae-fb70-46fe-8ca4-c918abe96a60 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:35.183996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e3997-4862-41f5-8285-8a9266207028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:35.184641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e3997-4862-41f5-8285-8a9266207028 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:35.184919 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e3997-4862-41f5-8285-8a9266207028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:35.185192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e3997-4862-41f5-8285-8a9266207028 admin admin] PATH_INFO: `/v3/projects/9e1d0638bd2d4078b8409acbe1e4335c/users/7ea2c080dd9f4e149d3041042a39ce66/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:35.202207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e3997-4862-41f5-8285-8a9266207028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e1d0638bd2d4078b8409acbe1e4335c, user_id=7ea2c080dd9f4e149d3041042a39ce66, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:35.205150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e3997-4862-41f5-8285-8a9266207028 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:35.218475 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1276/2549] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:35 2022] PUT /identity/v3/projects/9e1d0638bd2d4078b8409acbe1e4335c/users/7ea2c080dd9f4e149d3041042a39ce66/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:35.230029 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50b5f5-5b1d-4e28-9e47-d34c59936dbc None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:35.230376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50b5f5-5b1d-4e28-9e47-d34c59936dbc None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:35.230654 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50b5f5-5b1d-4e28-9e47-d34c59936dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:35.262268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-6c50b5f5-5b1d-4e28-9e47-d34c59936dbc None None] MFA Rules not processed for user `7ea2c080dd9f4e149d3041042a39ce66`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:35.263521 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6c50b5f5-5b1d-4e28-9e47-d34c59936dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:35.340649 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1274/2550] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:35.357018 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e3997-4862-41f5-8285-8a9266207028 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:35.358885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d62e3997-4862-41f5-8285-8a9266207028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:35.443943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e3997-4862-41f5-8285-8a9266207028 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:35.450464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982184d4-b75b-4fab-adef-d201ebb2f363 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:35.451307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982184d4-b75b-4fab-adef-d201ebb2f363 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:35.451520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982184d4-b75b-4fab-adef-d201ebb2f363 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:35.451671 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982184d4-b75b-4fab-adef-d201ebb2f363 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:35.462540 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982184d4-b75b-4fab-adef-d201ebb2f363 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:35.464323 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982184d4-b75b-4fab-adef-d201ebb2f363 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:35.475356 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1277/2551] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:35.724888 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50b5f5-5b1d-4e28-9e47-d34c59936dbc None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:35.793264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50b5f5-5b1d-4e28-9e47-d34c59936dbc None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:35.798165 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d2845-2c5e-4e5f-9e12-9b5840067ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:35.798944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d2845-2c5e-4e5f-9e12-9b5840067ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:35.799227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d2845-2c5e-4e5f-9e12-9b5840067ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:35.799501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d2845-2c5e-4e5f-9e12-9b5840067ae0 admin admin] PATH_INFO: `/v3/users/7ea2c080dd9f4e149d3041042a39ce66` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:35.800648 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d2845-2c5e-4e5f-9e12-9b5840067ae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ea2c080dd9f4e149d3041042a39ce66)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:35.802711 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d2845-2c5e-4e5f-9e12-9b5840067ae0 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:35.805430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:35.867658 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d76d2845-2c5e-4e5f-9e12-9b5840067ae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ea2c080dd9f4e149d3041042a39ce66'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:35.875488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d76d2845-2c5e-4e5f-9e12-9b5840067ae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ea2c080dd9f4e149d3041042a39ce66'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:35.890919 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d76d2845-2c5e-4e5f-9e12-9b5840067ae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ea2c080dd9f4e149d3041042a39ce66'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:35.891679 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d76d2845-2c5e-4e5f-9e12-9b5840067ae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ea2c080dd9f4e149d3041042a39ce66'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:35.892303 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d76d2845-2c5e-4e5f-9e12-9b5840067ae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ea2c080dd9f4e149d3041042a39ce66'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:35.906885 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1275/2552] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:35 2022] DELETE /identity/v3/users/7ea2c080dd9f4e149d3041042a39ce66 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:35.912060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f776c-f88c-416d-a035-38ed078e467f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:35.912344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f776c-f88c-416d-a035-38ed078e467f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:35.912616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f776c-f88c-416d-a035-38ed078e467f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:35.935190 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-078f776c-f88c-416d-a035-38ed078e467f None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:35.936540 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-078f776c-f88c-416d-a035-38ed078e467f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:36.008444 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1278/2553] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:27:36.023105 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d2845-2c5e-4e5f-9e12-9b5840067ae0 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:36.024706 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d76d2845-2c5e-4e5f-9e12-9b5840067ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:36.107516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d2845-2c5e-4e5f-9e12-9b5840067ae0 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:36.112375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db23bb2-413d-4be7-a3bc-d3d8c636b57b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:36.113120 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db23bb2-413d-4be7-a3bc-d3d8c636b57b service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:36.114047 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db23bb2-413d-4be7-a3bc-d3d8c636b57b service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:36.115150 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db23bb2-413d-4be7-a3bc-d3d8c636b57b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:36.134166 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db23bb2-413d-4be7-a3bc-d3d8c636b57b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:36.135750 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db23bb2-413d-4be7-a3bc-d3d8c636b57b service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:36.148780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1276/2554] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:27:36.466934 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f776c-f88c-416d-a035-38ed078e467f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:36.468478 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-078f776c-f88c-416d-a035-38ed078e467f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:36.482087 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f776c-f88c-416d-a035-38ed078e467f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:36.493365 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9f0c8-4c98-4f34-9c66-c47cf0a2a404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:36.494171 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9f0c8-4c98-4f34-9c66-c47cf0a2a404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:36.495117 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9f0c8-4c98-4f34-9c66-c47cf0a2a404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:36.496249 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9f0c8-4c98-4f34-9c66-c47cf0a2a404 admin admin] PATH_INFO: `/v3/projects/9e1d0638bd2d4078b8409acbe1e4335c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:36.498143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9f0c8-4c98-4f34-9c66-c47cf0a2a404 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e1d0638bd2d4078b8409acbe1e4335c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:36.501341 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9f0c8-4c98-4f34-9c66-c47cf0a2a404 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:36.557574 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:36.570564 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bae9f0c8-4c98-4f34-9c66-c47cf0a2a404 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e1d0638bd2d4078b8409acbe1e4335c', 'request_id': 'req-bae9f0c8-4c98-4f34-9c66-c47cf0a2a404'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:36.579058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bae9f0c8-4c98-4f34-9c66-c47cf0a2a404 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e1d0638bd2d4078b8409acbe1e4335c', 'request_id': 'req-bae9f0c8-4c98-4f34-9c66-c47cf0a2a404'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:36.589483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bae9f0c8-4c98-4f34-9c66-c47cf0a2a404 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e1d0638bd2d4078b8409acbe1e4335c', 'request_id': 'req-bae9f0c8-4c98-4f34-9c66-c47cf0a2a404'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:36.611968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bae9f0c8-4c98-4f34-9c66-c47cf0a2a404 admin admin] The token cache is being invalidate because project 9e1d0638bd2d4078b8409acbe1e4335c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:36.612309 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bae9f0c8-4c98-4f34-9c66-c47cf0a2a404 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:36.613021 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bae9f0c8-4c98-4f34-9c66-c47cf0a2a404 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:36.618460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1279/2555] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:36 2022] DELETE /identity/v3/projects/9e1d0638bd2d4078b8409acbe1e4335c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:27:36.620632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbc537-5174-4dc4-b2fb-7f2ef8c99000 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:36.621056 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbc537-5174-4dc4-b2fb-7f2ef8c99000 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:36.621631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbc537-5174-4dc4-b2fb-7f2ef8c99000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:36.644298 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-3abbc537-5174-4dc4-b2fb-7f2ef8c99000 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:36.646618 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3abbc537-5174-4dc4-b2fb-7f2ef8c99000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:36.747201 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1277/2556] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:36.755609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eebe92-0a1c-4aab-a120-2afe762f709f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:36.755933 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eebe92-0a1c-4aab-a120-2afe762f709f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:36.756205 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eebe92-0a1c-4aab-a120-2afe762f709f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:36.810398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f0eebe92-0a1c-4aab-a120-2afe762f709f None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:36.818809 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f0eebe92-0a1c-4aab-a120-2afe762f709f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:36.838094 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1280/2557] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:36.839381 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abbc537-5174-4dc4-b2fb-7f2ef8c99000 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:36.851565 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abbc537-5174-4dc4-b2fb-7f2ef8c99000 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:36.868025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c4ff8-7086-450b-8fa7-017e378883d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:36.871780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c4ff8-7086-450b-8fa7-017e378883d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:36.872103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c4ff8-7086-450b-8fa7-017e378883d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:36.872388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c4ff8-7086-450b-8fa7-017e378883d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:36.875250 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c4ff8-7086-450b-8fa7-017e378883d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:27:36.875597 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c4ff8-7086-450b-8fa7-017e378883d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:36.885109 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c4ff8-7086-450b-8fa7-017e378883d5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:36.916359 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eebe92-0a1c-4aab-a120-2afe762f709f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:36.918081 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f0eebe92-0a1c-4aab-a120-2afe762f709f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:36.922009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1278/2558] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:27:36.927074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c4ff8-7086-450b-8fa7-017e378883d5 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:36.938457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c4ff8-7086-450b-8fa7-017e378883d5 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:36.943722 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca724b48-50ef-4baa-96df-82bf255499de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:36.944364 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca724b48-50ef-4baa-96df-82bf255499de admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:36.944685 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca724b48-50ef-4baa-96df-82bf255499de admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:36.944982 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca724b48-50ef-4baa-96df-82bf255499de admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:36.946199 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca724b48-50ef-4baa-96df-82bf255499de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:36.948168 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca724b48-50ef-4baa-96df-82bf255499de admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:36.961260 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:36.982607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1279/2559] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:36 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:27:36.986711 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca724b48-50ef-4baa-96df-82bf255499de admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:36.992088 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eebe92-0a1c-4aab-a120-2afe762f709f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:37.045482 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22673f6a-fb3d-44c8-a962-7d72806467e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:37.046097 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22673f6a-fb3d-44c8-a962-7d72806467e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:37.046378 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22673f6a-fb3d-44c8-a962-7d72806467e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:37.046686 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22673f6a-fb3d-44c8-a962-7d72806467e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:37.048481 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-22673f6a-fb3d-44c8-a962-7d72806467e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:37.054549 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca724b48-50ef-4baa-96df-82bf255499de admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:37.059319 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc0485-d2a4-4a38-8cce-83b24ad500ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:37.059920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc0485-d2a4-4a38-8cce-83b24ad500ab admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:37.060198 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc0485-d2a4-4a38-8cce-83b24ad500ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:37.060484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc0485-d2a4-4a38-8cce-83b24ad500ab admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:37.061130 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc0485-d2a4-4a38-8cce-83b24ad500ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:37.063396 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc0485-d2a4-4a38-8cce-83b24ad500ab admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:37.087393 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1280/2560] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:36 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:27:37.091473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc0485-d2a4-4a38-8cce-83b24ad500ab admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:37.102727 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc0485-d2a4-4a38-8cce-83b24ad500ab admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:37.108038 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad7a00-f0ed-49cc-8c6f-e00a0f5600ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:37.108692 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad7a00-f0ed-49cc-8c6f-e00a0f5600ef admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:37.108985 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad7a00-f0ed-49cc-8c6f-e00a0f5600ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:37.109405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad7a00-f0ed-49cc-8c6f-e00a0f5600ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:37.110084 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad7a00-f0ed-49cc-8c6f-e00a0f5600ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:37.111813 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad7a00-f0ed-49cc-8c6f-e00a0f5600ef admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:37.121580 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22673f6a-fb3d-44c8-a962-7d72806467e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:37.123086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22673f6a-fb3d-44c8-a962-7d72806467e6 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:37.123615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1281/2561] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:27:37.127547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad7a00-f0ed-49cc-8c6f-e00a0f5600ef admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:37.134208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1281/2562] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:27:37.137773 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad7a00-f0ed-49cc-8c6f-e00a0f5600ef admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:37.142728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60078e-e3aa-4d40-ad9f-9e105c63d651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:37.143332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60078e-e3aa-4d40-ad9f-9e105c63d651 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:37.143616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60078e-e3aa-4d40-ad9f-9e105c63d651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:37.143888 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60078e-e3aa-4d40-ad9f-9e105c63d651 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:37.144529 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f60078e-e3aa-4d40-ad9f-9e105c63d651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:37.146787 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f60078e-e3aa-4d40-ad9f-9e105c63d651 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:37.159208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1282/2563] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:27:37.164206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22673f6a-fb3d-44c8-a962-7d72806467e6 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:37.175213 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22673f6a-fb3d-44c8-a962-7d72806467e6 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:37.180655 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3176675-a5ed-4375-8878-366bd5d8c8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:37.181328 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3176675-a5ed-4375-8878-366bd5d8c8d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:37.181593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3176675-a5ed-4375-8878-366bd5d8c8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:37.181869 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3176675-a5ed-4375-8878-366bd5d8c8d9 admin admin] PATH_INFO: `/v3/projects/f8b78286866242b9a0df4f9df450d440/users/80af01fcae154050a125f6ffe3bc7aac/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:37.198294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3176675-a5ed-4375-8878-366bd5d8c8d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b78286866242b9a0df4f9df450d440, user_id=80af01fcae154050a125f6ffe3bc7aac, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:37.201030 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3176675-a5ed-4375-8878-366bd5d8c8d9 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:37.211937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:37.217679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1282/2564] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:37 2022] PUT /identity/v3/projects/f8b78286866242b9a0df4f9df450d440/users/80af01fcae154050a125f6ffe3bc7aac/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:37.229017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca054533-ae4a-451d-bca9-99b3694a7184 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:37.229435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca054533-ae4a-451d-bca9-99b3694a7184 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:37.229716 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca054533-ae4a-451d-bca9-99b3694a7184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:37.260534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-ca054533-ae4a-451d-bca9-99b3694a7184 None None] MFA Rules not processed for user `80af01fcae154050a125f6ffe3bc7aac`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:37.261682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ca054533-ae4a-451d-bca9-99b3694a7184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:37.350856 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1283/2565] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:37.356522 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3176675-a5ed-4375-8878-366bd5d8c8d9 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:37.433470 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3176675-a5ed-4375-8878-366bd5d8c8d9 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:37.438261 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4893735-b632-4eed-98be-15a8bdffe597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:37.438889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4893735-b632-4eed-98be-15a8bdffe597 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:37.439172 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4893735-b632-4eed-98be-15a8bdffe597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:37.439442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4893735-b632-4eed-98be-15a8bdffe597 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:37.440078 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4893735-b632-4eed-98be-15a8bdffe597 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:37.442106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4893735-b632-4eed-98be-15a8bdffe597 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:37.477436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1283/2566] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:37 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:37.479475 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca054533-ae4a-451d-bca9-99b3694a7184 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:37.491992 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca054533-ae4a-451d-bca9-99b3694a7184 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:37.540911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:37.558582 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5be17-16df-4afd-82c7-09aa25309216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:37.559231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5be17-16df-4afd-82c7-09aa25309216 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:37.559522 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5be17-16df-4afd-82c7-09aa25309216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:37.559778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5be17-16df-4afd-82c7-09aa25309216 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:37.560410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5be17-16df-4afd-82c7-09aa25309216 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:37.566021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5be17-16df-4afd-82c7-09aa25309216 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:37.590430 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1284/2567] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:37 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:37.594592 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4893735-b632-4eed-98be-15a8bdffe597 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:37.614780 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4893735-b632-4eed-98be-15a8bdffe597 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:37.619595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7789d59-a7da-40f9-bb37-9d44680119af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:37.620335 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7789d59-a7da-40f9-bb37-9d44680119af admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:37.620608 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7789d59-a7da-40f9-bb37-9d44680119af admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:37.621076 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7789d59-a7da-40f9-bb37-9d44680119af admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:37.621752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7789d59-a7da-40f9-bb37-9d44680119af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:37.623501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7789d59-a7da-40f9-bb37-9d44680119af admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:37.634756 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1284/2568] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:37.639056 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5be17-16df-4afd-82c7-09aa25309216 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:37.649729 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5be17-16df-4afd-82c7-09aa25309216 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:37.654831 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d357333-87ab-4be6-bf79-acc793ea45e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:37.655583 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d357333-87ab-4be6-bf79-acc793ea45e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:37.655923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d357333-87ab-4be6-bf79-acc793ea45e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:37.656290 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d357333-87ab-4be6-bf79-acc793ea45e6 admin admin] PATH_INFO: `/v3/projects/118042ff6bc142a284ee3f57c800d779/users/bacb5d5ab5ae4d4a9fa2b1b85d1683ea/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:37.673278 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d357333-87ab-4be6-bf79-acc793ea45e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=118042ff6bc142a284ee3f57c800d779, user_id=bacb5d5ab5ae4d4a9fa2b1b85d1683ea, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:37.676267 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d357333-87ab-4be6-bf79-acc793ea45e6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:37.690394 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1285/2569] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:37 2022] PUT /identity/v3/projects/118042ff6bc142a284ee3f57c800d779/users/bacb5d5ab5ae4d4a9fa2b1b85d1683ea/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:27:37.694444 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7789d59-a7da-40f9-bb37-9d44680119af admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:37.705041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7789d59-a7da-40f9-bb37-9d44680119af admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:37.777655 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f247d-00a5-45a4-ad98-2c495227f62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:37.778300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f247d-00a5-45a4-ad98-2c495227f62f admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:37.778591 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f247d-00a5-45a4-ad98-2c495227f62f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:37.778894 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f247d-00a5-45a4-ad98-2c495227f62f admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:37.779545 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f247d-00a5-45a4-ad98-2c495227f62f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:37.781149 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f247d-00a5-45a4-ad98-2c495227f62f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:37.796405 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1285/2570] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:37.797694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d357333-87ab-4be6-bf79-acc793ea45e6 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:37.812646 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d357333-87ab-4be6-bf79-acc793ea45e6 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:37.820591 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7f958-997f-414d-9670-88693d751a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:37.821591 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7f958-997f-414d-9670-88693d751a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:37.822227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7f958-997f-414d-9670-88693d751a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:37.822570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7f958-997f-414d-9670-88693d751a73 admin admin] PATH_INFO: `/v3/projects/118042ff6bc142a284ee3f57c800d779/users/bacb5d5ab5ae4d4a9fa2b1b85d1683ea/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:37.826369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7f958-997f-414d-9670-88693d751a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=118042ff6bc142a284ee3f57c800d779, user_id=bacb5d5ab5ae4d4a9fa2b1b85d1683ea, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:37.831951 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7f958-997f-414d-9670-88693d751a73 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:37.849362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1286/2571] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:37 2022] PUT /identity/v3/projects/118042ff6bc142a284ee3f57c800d779/users/bacb5d5ab5ae4d4a9fa2b1b85d1683ea/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:37.861298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9599f2-57e5-45d5-8fa2-481e53f9d612 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:37.861849 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9599f2-57e5-45d5-8fa2-481e53f9d612 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:37.862118 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9599f2-57e5-45d5-8fa2-481e53f9d612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:37.896020 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ee9599f2-57e5-45d5-8fa2-481e53f9d612 None None] MFA Rules not processed for user `bacb5d5ab5ae4d4a9fa2b1b85d1683ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:37.897460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ee9599f2-57e5-45d5-8fa2-481e53f9d612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:37.992793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:38.011296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1286/2572] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:38.027171 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7f958-997f-414d-9670-88693d751a73 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:38.029170 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-16c7f958-997f-414d-9670-88693d751a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:38.141077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7f958-997f-414d-9670-88693d751a73 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:38.148553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1f965-e284-4969-9166-12f1a0863d65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:38.149593 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1f965-e284-4969-9166-12f1a0863d65 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:38.149890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1f965-e284-4969-9166-12f1a0863d65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:38.150189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1f965-e284-4969-9166-12f1a0863d65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:38.165641 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1f965-e284-4969-9166-12f1a0863d65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:38.167213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1f965-e284-4969-9166-12f1a0863d65 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:38.177900 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1287/2573] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:38.256462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9599f2-57e5-45d5-8fa2-481e53f9d612 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:38.334471 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9599f2-57e5-45d5-8fa2-481e53f9d612 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:38.339649 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c60f8f0-04b2-4e44-8f86-f25a0831b585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:38.340414 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60f8f0-04b2-4e44-8f86-f25a0831b585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:38.340765 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60f8f0-04b2-4e44-8f86-f25a0831b585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:38.341115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60f8f0-04b2-4e44-8f86-f25a0831b585 admin admin] PATH_INFO: `/v3/users/80af01fcae154050a125f6ffe3bc7aac` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:38.342789 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c60f8f0-04b2-4e44-8f86-f25a0831b585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80af01fcae154050a125f6ffe3bc7aac)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:38.344836 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c60f8f0-04b2-4e44-8f86-f25a0831b585 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:38.414555 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0c60f8f0-04b2-4e44-8f86-f25a0831b585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80af01fcae154050a125f6ffe3bc7aac'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:38.422192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0c60f8f0-04b2-4e44-8f86-f25a0831b585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80af01fcae154050a125f6ffe3bc7aac'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:38.434816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0c60f8f0-04b2-4e44-8f86-f25a0831b585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80af01fcae154050a125f6ffe3bc7aac'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:38.435556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0c60f8f0-04b2-4e44-8f86-f25a0831b585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80af01fcae154050a125f6ffe3bc7aac'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:38.444268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0c60f8f0-04b2-4e44-8f86-f25a0831b585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80af01fcae154050a125f6ffe3bc7aac'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:38.450693 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1287/2574] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:38 2022] DELETE /identity/v3/users/80af01fcae154050a125f6ffe3bc7aac => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:38.455548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4217b-ea6c-43ee-b329-4cc20aa410af None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:38.455829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4217b-ea6c-43ee-b329-4cc20aa410af None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:38.456130 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4217b-ea6c-43ee-b329-4cc20aa410af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:38.477229 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e7b4217b-ea6c-43ee-b329-4cc20aa410af None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:38.478440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e7b4217b-ea6c-43ee-b329-4cc20aa410af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:38.551953 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1288/2575] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:38.566531 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c60f8f0-04b2-4e44-8f86-f25a0831b585 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:38.568334 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0c60f8f0-04b2-4e44-8f86-f25a0831b585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:38.635516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c60f8f0-04b2-4e44-8f86-f25a0831b585 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:38.639952 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc7f8f-5d36-43e0-8fb6-9e3c1144ff7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:38.640620 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc7f8f-5d36-43e0-8fb6-9e3c1144ff7b service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:38.640962 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc7f8f-5d36-43e0-8fb6-9e3c1144ff7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:38.641241 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc7f8f-5d36-43e0-8fb6-9e3c1144ff7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:38.652616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc7f8f-5d36-43e0-8fb6-9e3c1144ff7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:38.654163 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc7f8f-5d36-43e0-8fb6-9e3c1144ff7b service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:38.666260 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1288/2576] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:38.998054 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4217b-ea6c-43ee-b329-4cc20aa410af None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:38.999581 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e7b4217b-ea6c-43ee-b329-4cc20aa410af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:39.018960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4217b-ea6c-43ee-b329-4cc20aa410af None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:39.025369 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f13ea-45c5-446d-b66c-fa9639fcd5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:39.026472 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f13ea-45c5-446d-b66c-fa9639fcd5ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:39.026931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f13ea-45c5-446d-b66c-fa9639fcd5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:39.027410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f13ea-45c5-446d-b66c-fa9639fcd5ca admin admin] PATH_INFO: `/v3/projects/f8b78286866242b9a0df4f9df450d440` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:39.029117 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f13ea-45c5-446d-b66c-fa9639fcd5ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8b78286866242b9a0df4f9df450d440)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:39.031138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f13ea-45c5-446d-b66c-fa9639fcd5ca admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:39.052473 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:39.093632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c51f13ea-45c5-446d-b66c-fa9639fcd5ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8b78286866242b9a0df4f9df450d440', 'request_id': 'req-c51f13ea-45c5-446d-b66c-fa9639fcd5ca'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:39.105315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c51f13ea-45c5-446d-b66c-fa9639fcd5ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8b78286866242b9a0df4f9df450d440', 'request_id': 'req-c51f13ea-45c5-446d-b66c-fa9639fcd5ca'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:39.111967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c51f13ea-45c5-446d-b66c-fa9639fcd5ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8b78286866242b9a0df4f9df450d440', 'request_id': 'req-c51f13ea-45c5-446d-b66c-fa9639fcd5ca'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:39.125141 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c51f13ea-45c5-446d-b66c-fa9639fcd5ca admin admin] The token cache is being invalidate because project f8b78286866242b9a0df4f9df450d440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:39.125486 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c51f13ea-45c5-446d-b66c-fa9639fcd5ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:39.126142 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c51f13ea-45c5-446d-b66c-fa9639fcd5ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:39.128603 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1289/2577] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:38 2022] DELETE /identity/v3/projects/f8b78286866242b9a0df4f9df450d440 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:39.133652 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc7f8f-5d36-43e0-8fb6-9e3c1144ff7b service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:39.149713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc7f8f-5d36-43e0-8fb6-9e3c1144ff7b service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:39.164278 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:39.227379 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c72ac-36a8-468c-ae33-611c65fbe59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:39.228034 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c72ac-36a8-468c-ae33-611c65fbe59a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:39.228316 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c72ac-36a8-468c-ae33-611c65fbe59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:39.228595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c72ac-36a8-468c-ae33-611c65fbe59a admin admin] PATH_INFO: `/v3/users/bacb5d5ab5ae4d4a9fa2b1b85d1683ea` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:39.229904 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c72ac-36a8-468c-ae33-611c65fbe59a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bacb5d5ab5ae4d4a9fa2b1b85d1683ea)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:39.231963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c72ac-36a8-468c-ae33-611c65fbe59a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:39.298039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-789c72ac-36a8-468c-ae33-611c65fbe59a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bacb5d5ab5ae4d4a9fa2b1b85d1683ea'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:39.311806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-789c72ac-36a8-468c-ae33-611c65fbe59a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bacb5d5ab5ae4d4a9fa2b1b85d1683ea'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:39.332417 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-789c72ac-36a8-468c-ae33-611c65fbe59a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bacb5d5ab5ae4d4a9fa2b1b85d1683ea'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:39.333303 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-789c72ac-36a8-468c-ae33-611c65fbe59a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bacb5d5ab5ae4d4a9fa2b1b85d1683ea'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:39.341493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-789c72ac-36a8-468c-ae33-611c65fbe59a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bacb5d5ab5ae4d4a9fa2b1b85d1683ea'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:39.350652 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1289/2578] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:39 2022] DELETE /identity/v3/users/bacb5d5ab5ae4d4a9fa2b1b85d1683ea => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:39.678242 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72ef24-19a6-4b96-a6bc-9ac14f1a084a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:39.678777 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72ef24-19a6-4b96-a6bc-9ac14f1a084a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:39.678953 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72ef24-19a6-4b96-a6bc-9ac14f1a084a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:39.704389 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8f72ef24-19a6-4b96-a6bc-9ac14f1a084a None None] MFA Rules not processed for user `ba58a4914ddf418db9e8b9dce5784cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:39.705780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8f72ef24-19a6-4b96-a6bc-9ac14f1a084a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:39.747451 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c72ac-36a8-468c-ae33-611c65fbe59a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:39.749760 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-789c72ac-36a8-468c-ae33-611c65fbe59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:39.774443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1290/2579] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 6 16:27:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:27:39.789452 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72ef24-19a6-4b96-a6bc-9ac14f1a084a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:39.791190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8f72ef24-19a6-4b96-a6bc-9ac14f1a084a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:39.842546 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c72ac-36a8-468c-ae33-611c65fbe59a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:39.847673 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d071fd-5fca-4ea4-8d81-33a3260a2c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:39.849864 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d071fd-5fca-4ea4-8d81-33a3260a2c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:39.850211 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d071fd-5fca-4ea4-8d81-33a3260a2c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:39.850557 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d071fd-5fca-4ea4-8d81-33a3260a2c4b admin admin] PATH_INFO: `/v3/projects/118042ff6bc142a284ee3f57c800d779` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:39.851931 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d071fd-5fca-4ea4-8d81-33a3260a2c4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=118042ff6bc142a284ee3f57c800d779)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:39.854000 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d071fd-5fca-4ea4-8d81-33a3260a2c4b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:39.908619 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72ef24-19a6-4b96-a6bc-9ac14f1a084a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:39.913671 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:39.925324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e5d071fd-5fca-4ea4-8d81-33a3260a2c4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '118042ff6bc142a284ee3f57c800d779', 'request_id': 'req-e5d071fd-5fca-4ea4-8d81-33a3260a2c4b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:39.938618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e5d071fd-5fca-4ea4-8d81-33a3260a2c4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '118042ff6bc142a284ee3f57c800d779', 'request_id': 'req-e5d071fd-5fca-4ea4-8d81-33a3260a2c4b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:39.947608 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e5d071fd-5fca-4ea4-8d81-33a3260a2c4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '118042ff6bc142a284ee3f57c800d779', 'request_id': 'req-e5d071fd-5fca-4ea4-8d81-33a3260a2c4b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:39.971012 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e5d071fd-5fca-4ea4-8d81-33a3260a2c4b admin admin] The token cache is being invalidate because project 118042ff6bc142a284ee3f57c800d779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:39.971649 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e5d071fd-5fca-4ea4-8d81-33a3260a2c4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:39.972737 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e5d071fd-5fca-4ea4-8d81-33a3260a2c4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:39.975718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1290/2580] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:39 2022] DELETE /identity/v3/projects/118042ff6bc142a284ee3f57c800d779 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:39.981130 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4bf49-6008-4047-a64b-3b3fdaa93af8 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:39.981912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4bf49-6008-4047-a64b-3b3fdaa93af8 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:39.982448 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4bf49-6008-4047-a64b-3b3fdaa93af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:39.986305 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:40.004047 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6254fb-8e52-4f6f-803d-04a5437d3221 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ece176d412574790a13d03dcca08f91a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:40.005702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6254fb-8e52-4f6f-803d-04a5437d3221 service swift] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:40.006518 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6254fb-8e52-4f6f-803d-04a5437d3221 service swift] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:40.006845 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6254fb-8e52-4f6f-803d-04a5437d3221 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:40.009695 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7b6254fb-8e52-4f6f-803d-04a5437d3221 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:40.012749 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-47c4bf49-6008-4047-a64b-3b3fdaa93af8 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:40.014644 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-47c4bf49-6008-4047-a64b-3b3fdaa93af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:40.096460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6254fb-8e52-4f6f-803d-04a5437d3221 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:40.100425 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6254fb-8e52-4f6f-803d-04a5437d3221 service swift] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:40.109462 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1291/2581] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 6 16:27:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 16:27:40.112020 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1291/2582] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:40.120021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d721f-8071-4f38-bf2b-bbc508f1e722 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:40.120513 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d721f-8071-4f38-bf2b-bbc508f1e722 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:40.120928 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d721f-8071-4f38-bf2b-bbc508f1e722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:40.148028 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0a5d721f-8071-4f38-bf2b-bbc508f1e722 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:40.149364 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0a5d721f-8071-4f38-bf2b-bbc508f1e722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:40.165059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1292/2583] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:40.169221 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4bf49-6008-4047-a64b-3b3fdaa93af8 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:40.179554 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4bf49-6008-4047-a64b-3b3fdaa93af8 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:40.185908 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761e31f-f4f7-4368-ac0f-613d84c35afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:40.186513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761e31f-f4f7-4368-ac0f-613d84c35afd admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:40.186862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761e31f-f4f7-4368-ac0f-613d84c35afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:40.187085 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761e31f-f4f7-4368-ac0f-613d84c35afd admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:40.187728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761e31f-f4f7-4368-ac0f-613d84c35afd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:27:40.188060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761e31f-f4f7-4368-ac0f-613d84c35afd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:40.189639 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761e31f-f4f7-4368-ac0f-613d84c35afd admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:40.205859 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1292/2584] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:27:40.208159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d721f-8071-4f38-bf2b-bbc508f1e722 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:40.218999 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d721f-8071-4f38-bf2b-bbc508f1e722 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:40.226068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be941793-0345-4881-971d-4a60e2e4b4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:40.227063 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be941793-0345-4881-971d-4a60e2e4b4f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:40.227574 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be941793-0345-4881-971d-4a60e2e4b4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:40.228044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be941793-0345-4881-971d-4a60e2e4b4f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:40.229232 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be941793-0345-4881-971d-4a60e2e4b4f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:40.232390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be941793-0345-4881-971d-4a60e2e4b4f5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:40.264746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1293/2585] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:40 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:40.271739 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761e31f-f4f7-4368-ac0f-613d84c35afd admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:40.285061 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761e31f-f4f7-4368-ac0f-613d84c35afd admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:40.377816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4046f01-aca5-4837-8fcf-c733d30aa86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:40.378573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4046f01-aca5-4837-8fcf-c733d30aa86d admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:40.379677 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4046f01-aca5-4837-8fcf-c733d30aa86d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:40.380685 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4046f01-aca5-4837-8fcf-c733d30aa86d admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:40.380893 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4046f01-aca5-4837-8fcf-c733d30aa86d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:40.383027 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4046f01-aca5-4837-8fcf-c733d30aa86d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:40.411087 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1293/2586] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:40 2022] POST /identity/v3/users => generated 283 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:40.415331 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be941793-0345-4881-971d-4a60e2e4b4f5 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:40.434682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be941793-0345-4881-971d-4a60e2e4b4f5 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:40.444873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3a299-c769-4cb4-81a8-7d0a66df5801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:40.452896 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3a299-c769-4cb4-81a8-7d0a66df5801 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:40.453664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3a299-c769-4cb4-81a8-7d0a66df5801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:40.454269 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3a299-c769-4cb4-81a8-7d0a66df5801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:40.455783 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe3a299-c769-4cb4-81a8-7d0a66df5801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:40.460329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe3a299-c769-4cb4-81a8-7d0a66df5801 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:40.477488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1294/2587] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:40.481527 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4046f01-aca5-4837-8fcf-c733d30aa86d admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:40.499084 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4046f01-aca5-4837-8fcf-c733d30aa86d admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:40.506244 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc58a8-09a5-459e-bf05-a512e34740b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:40.508074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc58a8-09a5-459e-bf05-a512e34740b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:40.508369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc58a8-09a5-459e-bf05-a512e34740b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:40.508670 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc58a8-09a5-459e-bf05-a512e34740b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:40.509909 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc58a8-09a5-459e-bf05-a512e34740b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:40.511373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc58a8-09a5-459e-bf05-a512e34740b6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:40.527989 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1294/2588] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:40.531855 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3a299-c769-4cb4-81a8-7d0a66df5801 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:40.545296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3a299-c769-4cb4-81a8-7d0a66df5801 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:40.552391 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d616e19-ccb7-4c42-a38d-50ca4b619aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:40.553142 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d616e19-ccb7-4c42-a38d-50ca4b619aba admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:40.553591 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d616e19-ccb7-4c42-a38d-50ca4b619aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:40.553870 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d616e19-ccb7-4c42-a38d-50ca4b619aba admin admin] PATH_INFO: `/v3/projects/24b53005cdcb42c5a5f89575d6b3fff7/users/10baadb7979345188987b72335062797/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:40.579427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d616e19-ccb7-4c42-a38d-50ca4b619aba admin admin] RBAC: Authorizing `identity:create_grant(project_id=24b53005cdcb42c5a5f89575d6b3fff7, user_id=10baadb7979345188987b72335062797, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:40.583652 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d616e19-ccb7-4c42-a38d-50ca4b619aba admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:40.617027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1295/2589] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:40 2022] PUT /identity/v3/projects/24b53005cdcb42c5a5f89575d6b3fff7/users/10baadb7979345188987b72335062797/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:27:40.622443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f91f7-8693-40b0-b8c6-b90fd1bc2e01 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:40.622921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f91f7-8693-40b0-b8c6-b90fd1bc2e01 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:40.623263 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f91f7-8693-40b0-b8c6-b90fd1bc2e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:40.668112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-659f91f7-8693-40b0-b8c6-b90fd1bc2e01 None None] MFA Rules not processed for user `10baadb7979345188987b72335062797`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:40.669839 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-659f91f7-8693-40b0-b8c6-b90fd1bc2e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:40.705792 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:40.807735 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1295/2590] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:40.813092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d616e19-ccb7-4c42-a38d-50ca4b619aba admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:40.906175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d616e19-ccb7-4c42-a38d-50ca4b619aba admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:40.911874 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171d168-eabc-44e6-b666-9d7f55114b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:40.913174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171d168-eabc-44e6-b666-9d7f55114b2f admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:40.913174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171d168-eabc-44e6-b666-9d7f55114b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:40.913174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171d168-eabc-44e6-b666-9d7f55114b2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:40.913881 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171d168-eabc-44e6-b666-9d7f55114b2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:40.917499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171d168-eabc-44e6-b666-9d7f55114b2f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:40.950700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1296/2591] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:40 2022] POST /identity/v3/projects => generated 367 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:40.954908 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f91f7-8693-40b0-b8c6-b90fd1bc2e01 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:40.969764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f91f7-8693-40b0-b8c6-b90fd1bc2e01 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:41.053079 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104a1c9-827e-4669-97b8-e8290439d96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:41.053696 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104a1c9-827e-4669-97b8-e8290439d96b admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:41.053974 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104a1c9-827e-4669-97b8-e8290439d96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:41.054244 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104a1c9-827e-4669-97b8-e8290439d96b admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:41.054923 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104a1c9-827e-4669-97b8-e8290439d96b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:41.056849 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104a1c9-827e-4669-97b8-e8290439d96b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:41.085637 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1296/2592] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:40 2022] POST /identity/v3/users => generated 290 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:41.086896 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171d168-eabc-44e6-b666-9d7f55114b2f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:41.098531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171d168-eabc-44e6-b666-9d7f55114b2f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:41.103814 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2dccf5-f337-42a7-a629-c0cfae81704b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:41.104459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dccf5-f337-42a7-a629-c0cfae81704b admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:41.104752 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dccf5-f337-42a7-a629-c0cfae81704b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:41.105028 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dccf5-f337-42a7-a629-c0cfae81704b admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:41.105735 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2dccf5-f337-42a7-a629-c0cfae81704b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:41.107641 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2dccf5-f337-42a7-a629-c0cfae81704b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:41.120171 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1297/2593] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:27:41.124464 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104a1c9-827e-4669-97b8-e8290439d96b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:41.135969 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104a1c9-827e-4669-97b8-e8290439d96b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:41.140802 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1856b-5765-4dcc-9b41-04bd55929e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:41.141458 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1856b-5765-4dcc-9b41-04bd55929e43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:41.141740 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1856b-5765-4dcc-9b41-04bd55929e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:41.142019 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1856b-5765-4dcc-9b41-04bd55929e43 admin admin] PATH_INFO: `/v3/projects/f1a2795afd554602aca0ba20a109db96/users/5781139f6f334e32b08c524ae4cf59f6/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:41.171960 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1856b-5765-4dcc-9b41-04bd55929e43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1a2795afd554602aca0ba20a109db96, user_id=5781139f6f334e32b08c524ae4cf59f6, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:41.174907 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1856b-5765-4dcc-9b41-04bd55929e43 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:41.194741 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1297/2594] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:41 2022] PUT /identity/v3/projects/f1a2795afd554602aca0ba20a109db96/users/5781139f6f334e32b08c524ae4cf59f6/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:41.197141 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2dccf5-f337-42a7-a629-c0cfae81704b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:41.215107 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2dccf5-f337-42a7-a629-c0cfae81704b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:41.216415 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:41.291031 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bc353-88fb-44b9-b6d8-4b59d992a3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:41.292143 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bc353-88fb-44b9-b6d8-4b59d992a3af admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:41.292434 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bc353-88fb-44b9-b6d8-4b59d992a3af admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:41.292712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bc353-88fb-44b9-b6d8-4b59d992a3af admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:41.293361 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bc353-88fb-44b9-b6d8-4b59d992a3af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:41.295077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bc353-88fb-44b9-b6d8-4b59d992a3af admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:41.310090 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1298/2595] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:41.311777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1856b-5765-4dcc-9b41-04bd55929e43 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:41.324645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1856b-5765-4dcc-9b41-04bd55929e43 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:41.329822 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041db339-1693-4674-ab9a-e799c6011d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:41.330527 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041db339-1693-4674-ab9a-e799c6011d19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:41.330844 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041db339-1693-4674-ab9a-e799c6011d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:41.331121 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041db339-1693-4674-ab9a-e799c6011d19 admin admin] PATH_INFO: `/v3/projects/f1a2795afd554602aca0ba20a109db96/users/5781139f6f334e32b08c524ae4cf59f6/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:41.333811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041db339-1693-4674-ab9a-e799c6011d19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1a2795afd554602aca0ba20a109db96, user_id=5781139f6f334e32b08c524ae4cf59f6, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:41.336536 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041db339-1693-4674-ab9a-e799c6011d19 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:41.352370 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1298/2596] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:41 2022] PUT /identity/v3/projects/f1a2795afd554602aca0ba20a109db96/users/5781139f6f334e32b08c524ae4cf59f6/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:41.364046 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b4575-8b4b-40d4-97d7-56c8a94ea6e3 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:41.364415 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b4575-8b4b-40d4-97d7-56c8a94ea6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:41.364804 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b4575-8b4b-40d4-97d7-56c8a94ea6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:41.414722 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-359b4575-8b4b-40d4-97d7-56c8a94ea6e3 None None] MFA Rules not processed for user `5781139f6f334e32b08c524ae4cf59f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:41.416895 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-359b4575-8b4b-40d4-97d7-56c8a94ea6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:41.528684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1299/2597] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:27:41.548534 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041db339-1693-4674-ab9a-e799c6011d19 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:41.551835 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-041db339-1693-4674-ab9a-e799c6011d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:41.636550 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041db339-1693-4674-ab9a-e799c6011d19 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:41.640410 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29174492-4057-4e28-9291-8254cb010bc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:41.641029 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29174492-4057-4e28-9291-8254cb010bc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:41.641345 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29174492-4057-4e28-9291-8254cb010bc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:41.641612 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29174492-4057-4e28-9291-8254cb010bc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:41.651939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29174492-4057-4e28-9291-8254cb010bc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:41.653378 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29174492-4057-4e28-9291-8254cb010bc1 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:41.675136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1299/2598] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:41.791402 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b4575-8b4b-40d4-97d7-56c8a94ea6e3 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:41.808340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b4575-8b4b-40d4-97d7-56c8a94ea6e3 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:41.812512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bd043-7dae-4a6a-8fea-8b6833ba7709 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:41.813212 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bd043-7dae-4a6a-8fea-8b6833ba7709 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:41.813796 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bd043-7dae-4a6a-8fea-8b6833ba7709 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:41.813880 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bd043-7dae-4a6a-8fea-8b6833ba7709 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:41.890520 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149bd043-7dae-4a6a-8fea-8b6833ba7709 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:41.892821 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149bd043-7dae-4a6a-8fea-8b6833ba7709 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:41.905467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1300/2599] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:47.147309 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29174492-4057-4e28-9291-8254cb010bc1 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:47.168413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:47.216584 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29174492-4057-4e28-9291-8254cb010bc1 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:47.221158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fc25d-9f53-41f4-853b-ab7b30332408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:47.221787 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fc25d-9f53-41f4-853b-ab7b30332408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:47.222053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fc25d-9f53-41f4-853b-ab7b30332408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:47.222324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fc25d-9f53-41f4-853b-ab7b30332408 admin admin] PATH_INFO: `/v3/users/10baadb7979345188987b72335062797` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:47.224487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452fc25d-9f53-41f4-853b-ab7b30332408 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10baadb7979345188987b72335062797)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:47.226398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452fc25d-9f53-41f4-853b-ab7b30332408 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:47.274225 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-452fc25d-9f53-41f4-853b-ab7b30332408 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10baadb7979345188987b72335062797'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:47.281185 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-452fc25d-9f53-41f4-853b-ab7b30332408 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10baadb7979345188987b72335062797'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:47.294005 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-452fc25d-9f53-41f4-853b-ab7b30332408 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10baadb7979345188987b72335062797'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:47.294719 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-452fc25d-9f53-41f4-853b-ab7b30332408 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10baadb7979345188987b72335062797'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:47.304604 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-452fc25d-9f53-41f4-853b-ab7b30332408 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10baadb7979345188987b72335062797'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:47.311113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1300/2600] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:47 2022] DELETE /identity/v3/users/10baadb7979345188987b72335062797 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:47.316476 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314169bd-d10c-4aa5-b549-b5edaa42d6bf None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:47.317071 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314169bd-d10c-4aa5-b549-b5edaa42d6bf None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:47.317292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314169bd-d10c-4aa5-b549-b5edaa42d6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:47.338371 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-314169bd-d10c-4aa5-b549-b5edaa42d6bf None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:47.339743 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-314169bd-d10c-4aa5-b549-b5edaa42d6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:47.359457 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:47.409504 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1301/2601] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:47.420292 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fc25d-9f53-41f4-853b-ab7b30332408 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:47.421623 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-452fc25d-9f53-41f4-853b-ab7b30332408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:47.481857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fc25d-9f53-41f4-853b-ab7b30332408 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:47.485712 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835417d-681c-418a-b1fc-dfe25578d146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:47.486877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835417d-681c-418a-b1fc-dfe25578d146 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:47.486877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835417d-681c-418a-b1fc-dfe25578d146 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:47.486877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835417d-681c-418a-b1fc-dfe25578d146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:47.496765 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835417d-681c-418a-b1fc-dfe25578d146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:47.498365 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835417d-681c-418a-b1fc-dfe25578d146 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:47.509158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1301/2602] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:47.745814 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314169bd-d10c-4aa5-b549-b5edaa42d6bf None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:47.758166 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314169bd-d10c-4aa5-b549-b5edaa42d6bf None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:47.763048 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7ed5f-c732-401a-9f9d-c24a105f624a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:47.763741 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7ed5f-c732-401a-9f9d-c24a105f624a service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:47.764015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7ed5f-c732-401a-9f9d-c24a105f624a service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:47.764488 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7ed5f-c732-401a-9f9d-c24a105f624a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:47.766470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9ab7ed5f-c732-401a-9f9d-c24a105f624a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:47.780169 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7ed5f-c732-401a-9f9d-c24a105f624a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:47.781611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7ed5f-c732-401a-9f9d-c24a105f624a service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:47.794308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1302/2603] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:47.821022 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835417d-681c-418a-b1fc-dfe25578d146 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:47.822727 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3835417d-681c-418a-b1fc-dfe25578d146 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:47.838839 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835417d-681c-418a-b1fc-dfe25578d146 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:47.844603 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87bb68-f661-4f6c-9f66-f771d9168c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:47.845386 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87bb68-f661-4f6c-9f66-f771d9168c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:47.845788 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87bb68-f661-4f6c-9f66-f771d9168c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:47.846132 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87bb68-f661-4f6c-9f66-f771d9168c2d admin admin] PATH_INFO: `/v3/projects/24b53005cdcb42c5a5f89575d6b3fff7` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:47.847562 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87bb68-f661-4f6c-9f66-f771d9168c2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=24b53005cdcb42c5a5f89575d6b3fff7)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:47.850677 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87bb68-f661-4f6c-9f66-f771d9168c2d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:47.892286 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:47.906984 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-1f87bb68-f661-4f6c-9f66-f771d9168c2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24b53005cdcb42c5a5f89575d6b3fff7', 'request_id': 'req-1f87bb68-f661-4f6c-9f66-f771d9168c2d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:47.915978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-1f87bb68-f661-4f6c-9f66-f771d9168c2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24b53005cdcb42c5a5f89575d6b3fff7', 'request_id': 'req-1f87bb68-f661-4f6c-9f66-f771d9168c2d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:47.922478 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-1f87bb68-f661-4f6c-9f66-f771d9168c2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24b53005cdcb42c5a5f89575d6b3fff7', 'request_id': 'req-1f87bb68-f661-4f6c-9f66-f771d9168c2d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:47.936893 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-1f87bb68-f661-4f6c-9f66-f771d9168c2d admin admin] The token cache is being invalidate because project 24b53005cdcb42c5a5f89575d6b3fff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:47.937248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-1f87bb68-f661-4f6c-9f66-f771d9168c2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:47.937897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-1f87bb68-f661-4f6c-9f66-f771d9168c2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:47.939601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1302/2604] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:47 2022] DELETE /identity/v3/projects/24b53005cdcb42c5a5f89575d6b3fff7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:47.945134 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7ed5f-c732-401a-9f9d-c24a105f624a service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:47.958110 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7ed5f-c732-401a-9f9d-c24a105f624a service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:48.039435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0cb45-fced-4366-b1c3-b143d8bfe093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:48.040262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0cb45-fced-4366-b1c3-b143d8bfe093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:48.040545 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0cb45-fced-4366-b1c3-b143d8bfe093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:48.040835 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0cb45-fced-4366-b1c3-b143d8bfe093 admin admin] PATH_INFO: `/v3/users/5781139f6f334e32b08c524ae4cf59f6` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:48.042114 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a0cb45-fced-4366-b1c3-b143d8bfe093 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5781139f6f334e32b08c524ae4cf59f6)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:48.044215 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a0cb45-fced-4366-b1c3-b143d8bfe093 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:48.107213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-68a0cb45-fced-4366-b1c3-b143d8bfe093 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5781139f6f334e32b08c524ae4cf59f6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:48.119548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-68a0cb45-fced-4366-b1c3-b143d8bfe093 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5781139f6f334e32b08c524ae4cf59f6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:48.139569 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-68a0cb45-fced-4366-b1c3-b143d8bfe093 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5781139f6f334e32b08c524ae4cf59f6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:48.141221 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-68a0cb45-fced-4366-b1c3-b143d8bfe093 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5781139f6f334e32b08c524ae4cf59f6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:48.147352 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-68a0cb45-fced-4366-b1c3-b143d8bfe093 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5781139f6f334e32b08c524ae4cf59f6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:48.168858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1303/2605] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:47 2022] DELETE /identity/v3/users/5781139f6f334e32b08c524ae4cf59f6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:48.512375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87bb68-f661-4f6c-9f66-f771d9168c2d admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:48.513973 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1f87bb68-f661-4f6c-9f66-f771d9168c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:48.528876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87bb68-f661-4f6c-9f66-f771d9168c2d admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:48.602900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a525fa-01f6-4c74-95ba-849f05e0bf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:48.603926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a525fa-01f6-4c74-95ba-849f05e0bf0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:48.603926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a525fa-01f6-4c74-95ba-849f05e0bf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:48.604139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a525fa-01f6-4c74-95ba-849f05e0bf0b admin admin] PATH_INFO: `/v3/projects/f1a2795afd554602aca0ba20a109db96` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:48.605215 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a525fa-01f6-4c74-95ba-849f05e0bf0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1a2795afd554602aca0ba20a109db96)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:48.607181 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a525fa-01f6-4c74-95ba-849f05e0bf0b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:48.667979 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-57a525fa-01f6-4c74-95ba-849f05e0bf0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1a2795afd554602aca0ba20a109db96', 'request_id': 'req-57a525fa-01f6-4c74-95ba-849f05e0bf0b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:48.674522 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-57a525fa-01f6-4c74-95ba-849f05e0bf0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1a2795afd554602aca0ba20a109db96', 'request_id': 'req-57a525fa-01f6-4c74-95ba-849f05e0bf0b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:48.680573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-57a525fa-01f6-4c74-95ba-849f05e0bf0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1a2795afd554602aca0ba20a109db96', 'request_id': 'req-57a525fa-01f6-4c74-95ba-849f05e0bf0b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:48.696699 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-57a525fa-01f6-4c74-95ba-849f05e0bf0b admin admin] The token cache is being invalidate because project f1a2795afd554602aca0ba20a109db96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:48.696914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-57a525fa-01f6-4c74-95ba-849f05e0bf0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:48.697609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-57a525fa-01f6-4c74-95ba-849f05e0bf0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:48.699561 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1303/2606] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:48 2022] DELETE /identity/v3/projects/f1a2795afd554602aca0ba20a109db96 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:48.705535 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb5355-d900-4d95-b99d-4370892d43fc None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:48.706324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb5355-d900-4d95-b99d-4370892d43fc None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:48.706324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb5355-d900-4d95-b99d-4370892d43fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:48.731113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0cdb5355-d900-4d95-b99d-4370892d43fc None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:48.732339 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0cdb5355-d900-4d95-b99d-4370892d43fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:48.762955 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:48.803624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1304/2607] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:48.811132 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938a69f-4c76-49ea-82c7-b12c9aa4fb17 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:48.811415 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938a69f-4c76-49ea-82c7-b12c9aa4fb17 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:48.811721 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938a69f-4c76-49ea-82c7-b12c9aa4fb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:48.832238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a938a69f-4c76-49ea-82c7-b12c9aa4fb17 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:48.833682 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a938a69f-4c76-49ea-82c7-b12c9aa4fb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:48.843396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1304/2608] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:48.848246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb5355-d900-4d95-b99d-4370892d43fc None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:48.858195 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb5355-d900-4d95-b99d-4370892d43fc None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:48.868388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b82e9-4d1c-42f5-bedb-dd60fcee1227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:48.869014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b82e9-4d1c-42f5-bedb-dd60fcee1227 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:48.869282 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b82e9-4d1c-42f5-bedb-dd60fcee1227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:48.869551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b82e9-4d1c-42f5-bedb-dd60fcee1227 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:48.870209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b82e9-4d1c-42f5-bedb-dd60fcee1227 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:27:48.870681 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b82e9-4d1c-42f5-bedb-dd60fcee1227 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:48.872394 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b82e9-4d1c-42f5-bedb-dd60fcee1227 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:48.886275 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1305/2609] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:27:48.890776 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938a69f-4c76-49ea-82c7-b12c9aa4fb17 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:48.901705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938a69f-4c76-49ea-82c7-b12c9aa4fb17 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:48.907311 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a69db-5648-4b19-94d7-5f6cb590a34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:48.907928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a69db-5648-4b19-94d7-5f6cb590a34e admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:48.908206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a69db-5648-4b19-94d7-5f6cb590a34e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:48.908474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a69db-5648-4b19-94d7-5f6cb590a34e admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:48.909142 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a69db-5648-4b19-94d7-5f6cb590a34e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:48.912309 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a69db-5648-4b19-94d7-5f6cb590a34e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:48.945245 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1305/2610] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:48 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:48.949834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b82e9-4d1c-42f5-bedb-dd60fcee1227 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:48.962439 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b82e9-4d1c-42f5-bedb-dd60fcee1227 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:49.024646 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe3b79-1640-49cd-846e-590870e7ba30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:49.025604 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe3b79-1640-49cd-846e-590870e7ba30 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:49.025894 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe3b79-1640-49cd-846e-590870e7ba30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:49.026153 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe3b79-1640-49cd-846e-590870e7ba30 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:49.026880 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe3b79-1640-49cd-846e-590870e7ba30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:49.028762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe3b79-1640-49cd-846e-590870e7ba30 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:49.052858 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1306/2611] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:48 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:27:49.060883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a69db-5648-4b19-94d7-5f6cb590a34e admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:49.074789 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a69db-5648-4b19-94d7-5f6cb590a34e admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:49.080068 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df2049-ccf1-4a2c-b9f3-e17715152f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:49.080678 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df2049-ccf1-4a2c-b9f3-e17715152f02 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:49.080953 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df2049-ccf1-4a2c-b9f3-e17715152f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:49.081224 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df2049-ccf1-4a2c-b9f3-e17715152f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:49.081874 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df2049-ccf1-4a2c-b9f3-e17715152f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:49.084600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df2049-ccf1-4a2c-b9f3-e17715152f02 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:49.096035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1306/2612] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:49.101713 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe3b79-1640-49cd-846e-590870e7ba30 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:49.125789 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe3b79-1640-49cd-846e-590870e7ba30 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:49.131626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb8f50-ef5d-42e2-b11e-703a572407be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:49.132233 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb8f50-ef5d-42e2-b11e-703a572407be admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:49.132526 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb8f50-ef5d-42e2-b11e-703a572407be admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:49.132794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb8f50-ef5d-42e2-b11e-703a572407be admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:49.133419 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb8f50-ef5d-42e2-b11e-703a572407be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:49.137941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb8f50-ef5d-42e2-b11e-703a572407be admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:49.150954 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1307/2613] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:49.152231 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df2049-ccf1-4a2c-b9f3-e17715152f02 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:49.164847 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df2049-ccf1-4a2c-b9f3-e17715152f02 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:49.170262 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246960a-4737-4628-a7c4-eb1c1c6e2a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:49.170980 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246960a-4737-4628-a7c4-eb1c1c6e2a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:49.171254 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246960a-4737-4628-a7c4-eb1c1c6e2a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:49.171525 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246960a-4737-4628-a7c4-eb1c1c6e2a3c admin admin] PATH_INFO: `/v3/projects/30583c60881b4975a80c79a795e8670b/users/1c4497d3b98541a28331bc8353b25aab/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:49.189021 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:49.197718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246960a-4737-4628-a7c4-eb1c1c6e2a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=30583c60881b4975a80c79a795e8670b, user_id=1c4497d3b98541a28331bc8353b25aab, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:49.202472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246960a-4737-4628-a7c4-eb1c1c6e2a3c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:49.230734 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1307/2614] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:49 2022] PUT /identity/v3/projects/30583c60881b4975a80c79a795e8670b/users/1c4497d3b98541a28331bc8353b25aab/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:49.248069 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1edf57-d3e5-4734-bf87-8cda1afeedc8 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:49.248952 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1edf57-d3e5-4734-bf87-8cda1afeedc8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:49.248952 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1edf57-d3e5-4734-bf87-8cda1afeedc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:49.289093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-7c1edf57-d3e5-4734-bf87-8cda1afeedc8 None None] MFA Rules not processed for user `1c4497d3b98541a28331bc8353b25aab`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:49.291124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7c1edf57-d3e5-4734-bf87-8cda1afeedc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:49.381053 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1308/2615] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:49.401884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246960a-4737-4628-a7c4-eb1c1c6e2a3c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:49.509897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246960a-4737-4628-a7c4-eb1c1c6e2a3c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:49.517674 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfec811-6d61-40ea-98b5-505ff79fe92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:49.518401 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfec811-6d61-40ea-98b5-505ff79fe92d admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:49.518588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfec811-6d61-40ea-98b5-505ff79fe92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:49.518902 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfec811-6d61-40ea-98b5-505ff79fe92d admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:49.520194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfec811-6d61-40ea-98b5-505ff79fe92d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:49.522339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfec811-6d61-40ea-98b5-505ff79fe92d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:49.567198 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1308/2616] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:49 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:49.570186 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1edf57-d3e5-4734-bf87-8cda1afeedc8 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:49.584076 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1edf57-d3e5-4734-bf87-8cda1afeedc8 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:49.626183 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:49.686226 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c5bd7-0c43-44b4-8997-50a6d54a59a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:49.687316 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c5bd7-0c43-44b4-8997-50a6d54a59a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:49.687829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c5bd7-0c43-44b4-8997-50a6d54a59a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:49.688324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c5bd7-0c43-44b4-8997-50a6d54a59a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:49.689512 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c5bd7-0c43-44b4-8997-50a6d54a59a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:49.694024 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c5bd7-0c43-44b4-8997-50a6d54a59a0 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:49.726863 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1309/2617] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:49 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:27:49.733115 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfec811-6d61-40ea-98b5-505ff79fe92d admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:49.753607 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfec811-6d61-40ea-98b5-505ff79fe92d admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:49.762458 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f2099-5306-450a-94a4-b9442b0f3851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:49.763408 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f2099-5306-450a-94a4-b9442b0f3851 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:49.763889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f2099-5306-450a-94a4-b9442b0f3851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:49.764352 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f2099-5306-450a-94a4-b9442b0f3851 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:49.765538 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f2099-5306-450a-94a4-b9442b0f3851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:49.768658 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f2099-5306-450a-94a4-b9442b0f3851 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:49.786447 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1309/2618] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:49.791054 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c5bd7-0c43-44b4-8997-50a6d54a59a0 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:49.803328 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c5bd7-0c43-44b4-8997-50a6d54a59a0 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:49.808916 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e41066-cc84-41b5-b541-88018c563b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:49.809541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e41066-cc84-41b5-b541-88018c563b35 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:49.809826 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e41066-cc84-41b5-b541-88018c563b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:49.810124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e41066-cc84-41b5-b541-88018c563b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:49.810780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e41066-cc84-41b5-b541-88018c563b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:49.812508 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e41066-cc84-41b5-b541-88018c563b35 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:49.824672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1310/2619] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:49.829963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f2099-5306-450a-94a4-b9442b0f3851 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:49.850942 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f2099-5306-450a-94a4-b9442b0f3851 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:49.860070 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad3415-cad3-47be-8c38-e8a26310f082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:49.861400 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad3415-cad3-47be-8c38-e8a26310f082 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:49.862045 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad3415-cad3-47be-8c38-e8a26310f082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:49.862767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad3415-cad3-47be-8c38-e8a26310f082 admin admin] PATH_INFO: `/v3/projects/f7705615c50244f1a1ae29e38afe5a7e/users/1c5b1e9a6d2c4eb3928f137df7fc7193/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:49.889659 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad3415-cad3-47be-8c38-e8a26310f082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7705615c50244f1a1ae29e38afe5a7e, user_id=1c5b1e9a6d2c4eb3928f137df7fc7193, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:49.892459 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad3415-cad3-47be-8c38-e8a26310f082 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:49.918451 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1310/2620] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:49 2022] PUT /identity/v3/projects/f7705615c50244f1a1ae29e38afe5a7e/users/1c5b1e9a6d2c4eb3928f137df7fc7193/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:49.920541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8b323-83ab-43dd-84fa-fcbc25a25d8a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:49.920949 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8b323-83ab-43dd-84fa-fcbc25a25d8a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:49.921298 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8b323-83ab-43dd-84fa-fcbc25a25d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:49.954911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-52f8b323-83ab-43dd-84fa-fcbc25a25d8a None None] MFA Rules not processed for user `1c5b1e9a6d2c4eb3928f137df7fc7193`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:49.956381 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-52f8b323-83ab-43dd-84fa-fcbc25a25d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:50.052828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1311/2621] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:27:50.057431 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad3415-cad3-47be-8c38-e8a26310f082 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:50.154444 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad3415-cad3-47be-8c38-e8a26310f082 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:50.159917 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00c40a-48b6-4136-ac70-a2780f009ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:50.161238 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00c40a-48b6-4136-ac70-a2780f009ea3 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:50.161460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00c40a-48b6-4136-ac70-a2780f009ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:50.161824 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00c40a-48b6-4136-ac70-a2780f009ea3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:50.162950 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00c40a-48b6-4136-ac70-a2780f009ea3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:50.165165 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00c40a-48b6-4136-ac70-a2780f009ea3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:50.200363 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1311/2622] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:50 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:50.205217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8b323-83ab-43dd-84fa-fcbc25a25d8a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:50.218886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8b323-83ab-43dd-84fa-fcbc25a25d8a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:50.324927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf884f-5a8f-4604-8e28-05adb967ebfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:50.325656 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf884f-5a8f-4604-8e28-05adb967ebfc admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:50.326481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf884f-5a8f-4604-8e28-05adb967ebfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:50.326481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf884f-5a8f-4604-8e28-05adb967ebfc admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:50.327618 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf884f-5a8f-4604-8e28-05adb967ebfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:50.330311 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf884f-5a8f-4604-8e28-05adb967ebfc admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:50.357482 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1312/2623] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:50 2022] POST /identity/v3/users => generated 298 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:50.361720 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00c40a-48b6-4136-ac70-a2780f009ea3 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:50.376295 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00c40a-48b6-4136-ac70-a2780f009ea3 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:50.383547 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19d47b-623c-4f9b-ad56-d21e773c3959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:50.384299 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19d47b-623c-4f9b-ad56-d21e773c3959 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:50.384668 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19d47b-623c-4f9b-ad56-d21e773c3959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:50.385032 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19d47b-623c-4f9b-ad56-d21e773c3959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:50.386517 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19d47b-623c-4f9b-ad56-d21e773c3959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:50.388743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19d47b-623c-4f9b-ad56-d21e773c3959 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:50.405472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1312/2624] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:50.409878 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf884f-5a8f-4604-8e28-05adb967ebfc admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:50.420962 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf884f-5a8f-4604-8e28-05adb967ebfc admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:50.427214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e6066-73dc-4949-b784-5da5e80b3728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:50.427900 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e6066-73dc-4949-b784-5da5e80b3728 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:50.428175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e6066-73dc-4949-b784-5da5e80b3728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:50.428445 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e6066-73dc-4949-b784-5da5e80b3728 admin admin] PATH_INFO: `/v3/projects/96718d133e5047bab511086a48bc1d31/users/3c00ac1788af4298a82c141455e2c7f4/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:50.450999 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e6066-73dc-4949-b784-5da5e80b3728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96718d133e5047bab511086a48bc1d31, user_id=3c00ac1788af4298a82c141455e2c7f4, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:50.454212 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e6066-73dc-4949-b784-5da5e80b3728 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:50.467513 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:50.479521 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1313/2625] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:50 2022] PUT /identity/v3/projects/96718d133e5047bab511086a48bc1d31/users/3c00ac1788af4298a82c141455e2c7f4/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:50.483182 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19d47b-623c-4f9b-ad56-d21e773c3959 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:50.495606 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19d47b-623c-4f9b-ad56-d21e773c3959 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:50.519452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:50.612989 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca33fe-99e5-4474-ac35-ded4303aa805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:50.615329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca33fe-99e5-4474-ac35-ded4303aa805 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:50.615616 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca33fe-99e5-4474-ac35-ded4303aa805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:50.615892 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca33fe-99e5-4474-ac35-ded4303aa805 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:50.618406 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca33fe-99e5-4474-ac35-ded4303aa805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:50.620131 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca33fe-99e5-4474-ac35-ded4303aa805 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:50.632555 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1313/2626] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:50.637008 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e6066-73dc-4949-b784-5da5e80b3728 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:50.651087 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e6066-73dc-4949-b784-5da5e80b3728 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:50.657757 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077e7f1-dc79-4daa-98fa-d71967d32211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:50.658909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077e7f1-dc79-4daa-98fa-d71967d32211 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:50.659403 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077e7f1-dc79-4daa-98fa-d71967d32211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:50.659884 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077e7f1-dc79-4daa-98fa-d71967d32211 admin admin] PATH_INFO: `/v3/projects/96718d133e5047bab511086a48bc1d31/users/3c00ac1788af4298a82c141455e2c7f4/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:50.663700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077e7f1-dc79-4daa-98fa-d71967d32211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96718d133e5047bab511086a48bc1d31, user_id=3c00ac1788af4298a82c141455e2c7f4, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:50.669329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077e7f1-dc79-4daa-98fa-d71967d32211 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:50.688688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1314/2627] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:50 2022] PUT /identity/v3/projects/96718d133e5047bab511086a48bc1d31/users/3c00ac1788af4298a82c141455e2c7f4/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:50.709694 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e3b34-3dae-441e-8e46-9bfef67327c5 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:50.710218 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e3b34-3dae-441e-8e46-9bfef67327c5 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:50.710716 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e3b34-3dae-441e-8e46-9bfef67327c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:50.752861 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a45e3b34-3dae-441e-8e46-9bfef67327c5 None None] MFA Rules not processed for user `3c00ac1788af4298a82c141455e2c7f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:50.754995 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a45e3b34-3dae-441e-8e46-9bfef67327c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:50.855103 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1314/2628] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:50.867927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077e7f1-dc79-4daa-98fa-d71967d32211 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:50.870122 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9077e7f1-dc79-4daa-98fa-d71967d32211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:50.953561 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077e7f1-dc79-4daa-98fa-d71967d32211 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:50.959162 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c412a01-e273-4637-985f-793ad335b4e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:50.959880 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c412a01-e273-4637-985f-793ad335b4e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:50.960400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c412a01-e273-4637-985f-793ad335b4e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:50.960771 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c412a01-e273-4637-985f-793ad335b4e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:51.055983 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c412a01-e273-4637-985f-793ad335b4e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:51.057453 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c412a01-e273-4637-985f-793ad335b4e2 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:51.068149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1315/2629] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:27:52.071791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e3b34-3dae-441e-8e46-9bfef67327c5 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:52.073767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a45e3b34-3dae-441e-8e46-9bfef67327c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:52.154930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e3b34-3dae-441e-8e46-9bfef67327c5 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:52.159932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b666fc-3585-4d16-a539-cdff904a7429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:52.160770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b666fc-3585-4d16-a539-cdff904a7429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:52.161147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b666fc-3585-4d16-a539-cdff904a7429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:52.161521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b666fc-3585-4d16-a539-cdff904a7429 admin admin] PATH_INFO: `/v3/users/95ac6a289fb345d3a98871f5e11f9414` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:52.163065 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b666fc-3585-4d16-a539-cdff904a7429 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95ac6a289fb345d3a98871f5e11f9414)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:52.165260 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b666fc-3585-4d16-a539-cdff904a7429 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:52.217257 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:52.231082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d5b666fc-3585-4d16-a539-cdff904a7429 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95ac6a289fb345d3a98871f5e11f9414'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:52.243671 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d5b666fc-3585-4d16-a539-cdff904a7429 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95ac6a289fb345d3a98871f5e11f9414'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:52.264628 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d5b666fc-3585-4d16-a539-cdff904a7429 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95ac6a289fb345d3a98871f5e11f9414'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:52.265697 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d5b666fc-3585-4d16-a539-cdff904a7429 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95ac6a289fb345d3a98871f5e11f9414'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:52.279467 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d5b666fc-3585-4d16-a539-cdff904a7429 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95ac6a289fb345d3a98871f5e11f9414'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:52.287887 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1315/2630] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:52 2022] DELETE /identity/v3/users/95ac6a289fb345d3a98871f5e11f9414 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:52.499160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c412a01-e273-4637-985f-793ad335b4e2 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:52.516684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c412a01-e273-4637-985f-793ad335b4e2 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:52.593045 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787619b-20b8-46c1-a625-5945b2591a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:52.593806 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787619b-20b8-46c1-a625-5945b2591a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:52.594282 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787619b-20b8-46c1-a625-5945b2591a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:52.594637 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787619b-20b8-46c1-a625-5945b2591a89 admin admin] PATH_INFO: `/v3/projects/d306388a489a4b42a98a85e03968dba7` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:52.595915 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787619b-20b8-46c1-a625-5945b2591a89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d306388a489a4b42a98a85e03968dba7)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:52.598208 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787619b-20b8-46c1-a625-5945b2591a89 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:52.662487 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:52.668327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2787619b-20b8-46c1-a625-5945b2591a89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd306388a489a4b42a98a85e03968dba7', 'request_id': 'req-2787619b-20b8-46c1-a625-5945b2591a89'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:52.683837 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2787619b-20b8-46c1-a625-5945b2591a89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd306388a489a4b42a98a85e03968dba7', 'request_id': 'req-2787619b-20b8-46c1-a625-5945b2591a89'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:52.692049 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2787619b-20b8-46c1-a625-5945b2591a89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd306388a489a4b42a98a85e03968dba7', 'request_id': 'req-2787619b-20b8-46c1-a625-5945b2591a89'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:52.705138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2787619b-20b8-46c1-a625-5945b2591a89 admin admin] The token cache is being invalidate because project d306388a489a4b42a98a85e03968dba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:52.705503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2787619b-20b8-46c1-a625-5945b2591a89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:52.706272 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2787619b-20b8-46c1-a625-5945b2591a89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:52.707833 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1316/2631] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:52 2022] DELETE /identity/v3/projects/d306388a489a4b42a98a85e03968dba7 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:52.713255 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db0424-3517-43cf-b3d9-1a14d927bd9d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:52.713656 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db0424-3517-43cf-b3d9-1a14d927bd9d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:52.713852 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db0424-3517-43cf-b3d9-1a14d927bd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:52.736159 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-95db0424-3517-43cf-b3d9-1a14d927bd9d None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:52.737469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-95db0424-3517-43cf-b3d9-1a14d927bd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:52.760903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787619b-20b8-46c1-a625-5945b2591a89 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:52.763154 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2787619b-20b8-46c1-a625-5945b2591a89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:52.808151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1316/2632] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:52.821968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8718cf-8afa-4067-baaf-36e22b3abccf None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:52.821968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8718cf-8afa-4067-baaf-36e22b3abccf None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:52.821968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8718cf-8afa-4067-baaf-36e22b3abccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:52.832749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787619b-20b8-46c1-a625-5945b2591a89 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:52.836734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23963af0-4d8b-4734-b37b-d139d6064134 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:52.837354 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23963af0-4d8b-4734-b37b-d139d6064134 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:52.837637 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23963af0-4d8b-4734-b37b-d139d6064134 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:52.837921 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23963af0-4d8b-4734-b37b-d139d6064134 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:52.838912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4c8718cf-8afa-4067-baaf-36e22b3abccf None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:52.839808 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-23963af0-4d8b-4734-b37b-d139d6064134 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:52.840134 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4c8718cf-8afa-4067-baaf-36e22b3abccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:52.849804 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1317/2633] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:27:52.853719 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8718cf-8afa-4067-baaf-36e22b3abccf None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:52.864050 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8718cf-8afa-4067-baaf-36e22b3abccf None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:52.868925 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc440b7-aa51-462c-87c2-617a7d0fc6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:52.869528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc440b7-aa51-462c-87c2-617a7d0fc6f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:52.869808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc440b7-aa51-462c-87c2-617a7d0fc6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:52.870083 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc440b7-aa51-462c-87c2-617a7d0fc6f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:52.870858 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc440b7-aa51-462c-87c2-617a7d0fc6f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:27:52.871067 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc440b7-aa51-462c-87c2-617a7d0fc6f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:52.872705 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc440b7-aa51-462c-87c2-617a7d0fc6f5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:52.887305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1318/2634] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:27:52.891505 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc440b7-aa51-462c-87c2-617a7d0fc6f5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:52.901832 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc440b7-aa51-462c-87c2-617a7d0fc6f5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:52.906558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b766f0f-605a-4458-ba71-f7ac7ac97622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:52.907191 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b766f0f-605a-4458-ba71-f7ac7ac97622 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:52.907468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23963af0-4d8b-4734-b37b-d139d6064134 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:52.907595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b766f0f-605a-4458-ba71-f7ac7ac97622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:52.907761 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b766f0f-605a-4458-ba71-f7ac7ac97622 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:52.908396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b766f0f-605a-4458-ba71-f7ac7ac97622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:52.908928 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23963af0-4d8b-4734-b37b-d139d6064134 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:52.911084 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b766f0f-605a-4458-ba71-f7ac7ac97622 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:52.920842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1317/2635] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:52.947663 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1319/2636] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:52 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:27:52.951966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23963af0-4d8b-4734-b37b-d139d6064134 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:52.963688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23963af0-4d8b-4734-b37b-d139d6064134 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:53.032100 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b9393-cec7-475a-92a2-ee79eb927db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:53.032719 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b9393-cec7-475a-92a2-ee79eb927db8 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:53.033002 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b9393-cec7-475a-92a2-ee79eb927db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:53.033276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b9393-cec7-475a-92a2-ee79eb927db8 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:53.033961 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b9393-cec7-475a-92a2-ee79eb927db8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:53.036634 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b9393-cec7-475a-92a2-ee79eb927db8 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:53.061217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1318/2637] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:52 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:53.065586 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b766f0f-605a-4458-ba71-f7ac7ac97622 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:53.077576 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b766f0f-605a-4458-ba71-f7ac7ac97622 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:53.084918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4089c5-7983-4636-9250-42763ed3d1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:53.085534 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4089c5-7983-4636-9250-42763ed3d1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:53.085798 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4089c5-7983-4636-9250-42763ed3d1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:53.086098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4089c5-7983-4636-9250-42763ed3d1c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:53.086732 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4089c5-7983-4636-9250-42763ed3d1c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:53.088407 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4089c5-7983-4636-9250-42763ed3d1c5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:53.099607 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1320/2638] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:53.103823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b9393-cec7-475a-92a2-ee79eb927db8 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:53.116447 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b9393-cec7-475a-92a2-ee79eb927db8 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:53.121455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92de858-c6b8-4a5e-81b0-c9a462816e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:53.122090 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92de858-c6b8-4a5e-81b0-c9a462816e19 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:53.122377 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92de858-c6b8-4a5e-81b0-c9a462816e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:53.122693 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92de858-c6b8-4a5e-81b0-c9a462816e19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:53.123337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92de858-c6b8-4a5e-81b0-c9a462816e19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:53.124944 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92de858-c6b8-4a5e-81b0-c9a462816e19 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:53.136133 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1319/2639] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:53.141976 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4089c5-7983-4636-9250-42763ed3d1c5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:53.156193 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4089c5-7983-4636-9250-42763ed3d1c5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:53.161141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4747e-4ca9-44b1-b446-a4dd39f5f11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:53.161841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4747e-4ca9-44b1-b446-a4dd39f5f11b admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:53.162107 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4747e-4ca9-44b1-b446-a4dd39f5f11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:53.162382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4747e-4ca9-44b1-b446-a4dd39f5f11b admin admin] PATH_INFO: `/v3/projects/0eeea71a0c344b28b4e7797dea4cc338/users/916cfaffed5b46f19db6b65acee0552b/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:53.180584 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4747e-4ca9-44b1-b446-a4dd39f5f11b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eeea71a0c344b28b4e7797dea4cc338, user_id=916cfaffed5b46f19db6b65acee0552b, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:53.184108 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4747e-4ca9-44b1-b446-a4dd39f5f11b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:53.202084 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1321/2640] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:53 2022] PUT /identity/v3/projects/0eeea71a0c344b28b4e7797dea4cc338/users/916cfaffed5b46f19db6b65acee0552b/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:53.215646 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b047f4-e5d0-48c4-9aad-44fc5aa7a11a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:53.215926 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b047f4-e5d0-48c4-9aad-44fc5aa7a11a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:53.216222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b047f4-e5d0-48c4-9aad-44fc5aa7a11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:53.226694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:53.258854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-91b047f4-e5d0-48c4-9aad-44fc5aa7a11a None None] MFA Rules not processed for user `916cfaffed5b46f19db6b65acee0552b`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:53.260339 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-91b047f4-e5d0-48c4-9aad-44fc5aa7a11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:53.354910 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1320/2641] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:53.359703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4747e-4ca9-44b1-b446-a4dd39f5f11b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:53.408234 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:53.433694 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4747e-4ca9-44b1-b446-a4dd39f5f11b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:53.438527 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837712d-d8f9-4c4f-946c-aa11b6ef2789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:53.439165 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837712d-d8f9-4c4f-946c-aa11b6ef2789 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:53.439442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837712d-d8f9-4c4f-946c-aa11b6ef2789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:53.439713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837712d-d8f9-4c4f-946c-aa11b6ef2789 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:53.440346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837712d-d8f9-4c4f-946c-aa11b6ef2789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:53.443491 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837712d-d8f9-4c4f-946c-aa11b6ef2789 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:53.475510 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1322/2642] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:53 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:27:53.479839 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b047f4-e5d0-48c4-9aad-44fc5aa7a11a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:53.493110 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b047f4-e5d0-48c4-9aad-44fc5aa7a11a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:53.535577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837712d-d8f9-4c4f-946c-aa11b6ef2789 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:53.560349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e1794-8941-42d1-bae2-c1871927b1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:53.561072 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e1794-8941-42d1-bae2-c1871927b1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:53.561365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e1794-8941-42d1-bae2-c1871927b1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:53.561642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e1794-8941-42d1-bae2-c1871927b1a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:53.562292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e1794-8941-42d1-bae2-c1871927b1a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:53.564313 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e1794-8941-42d1-bae2-c1871927b1a9 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:53.593535 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1321/2643] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:53 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:27:53.597584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e1794-8941-42d1-bae2-c1871927b1a9 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:53.614295 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e1794-8941-42d1-bae2-c1871927b1a9 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:53.618914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837712d-d8f9-4c4f-946c-aa11b6ef2789 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:53.620554 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e25e5-9906-4043-a704-05c647107871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:53.621680 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e25e5-9906-4043-a704-05c647107871 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:53.622106 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e25e5-9906-4043-a704-05c647107871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:53.622536 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e25e5-9906-4043-a704-05c647107871 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:53.622838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2705a-56ae-44bd-9fe2-513509367a75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:53.623482 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2705a-56ae-44bd-9fe2-513509367a75 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:53.623677 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e25e5-9906-4043-a704-05c647107871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:53.623829 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2705a-56ae-44bd-9fe2-513509367a75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:53.624003 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2705a-56ae-44bd-9fe2-513509367a75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:53.625874 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-56c2705a-56ae-44bd-9fe2-513509367a75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:53.626058 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e25e5-9906-4043-a704-05c647107871 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:53.638111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1322/2644] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:27:53.642407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e25e5-9906-4043-a704-05c647107871 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:53.652842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e25e5-9906-4043-a704-05c647107871 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:53.657591 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171185e-41a0-4782-b094-3ac57f24b728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:53.658242 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171185e-41a0-4782-b094-3ac57f24b728 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:53.658522 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171185e-41a0-4782-b094-3ac57f24b728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:53.658827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171185e-41a0-4782-b094-3ac57f24b728 admin admin] PATH_INFO: `/v3/projects/f71b71b774714529a2b47437e4f4f38d/users/4206f7f253cd42a29847e169022a2d36/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:53.674788 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3171185e-41a0-4782-b094-3ac57f24b728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f71b71b774714529a2b47437e4f4f38d, user_id=4206f7f253cd42a29847e169022a2d36, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:53.678134 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3171185e-41a0-4782-b094-3ac57f24b728 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:53.693681 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1323/2645] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:53 2022] PUT /identity/v3/projects/f71b71b774714529a2b47437e4f4f38d/users/4206f7f253cd42a29847e169022a2d36/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:27:53.696781 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2705a-56ae-44bd-9fe2-513509367a75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:53.698262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171185e-41a0-4782-b094-3ac57f24b728 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:53.698451 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2705a-56ae-44bd-9fe2-513509367a75 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:53.709911 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1323/2646] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:27:53.753461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:53.772068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171185e-41a0-4782-b094-3ac57f24b728 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:53.777843 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f212e33-53ed-4855-9950-f7195b83dc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:53.778463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f212e33-53ed-4855-9950-f7195b83dc40 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:53.778873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f212e33-53ed-4855-9950-f7195b83dc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:53.779074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f212e33-53ed-4855-9950-f7195b83dc40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:53.779655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f212e33-53ed-4855-9950-f7195b83dc40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:53.781274 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f212e33-53ed-4855-9950-f7195b83dc40 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:53.797132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1324/2647] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:53.798552 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2705a-56ae-44bd-9fe2-513509367a75 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:53.814042 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2705a-56ae-44bd-9fe2-513509367a75 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:53.820054 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd7c25-2986-4f95-baae-028cd38c577d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:53.820714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd7c25-2986-4f95-baae-028cd38c577d admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:53.821142 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd7c25-2986-4f95-baae-028cd38c577d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:53.821416 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd7c25-2986-4f95-baae-028cd38c577d admin admin] PATH_INFO: `/v3/projects/f71b71b774714529a2b47437e4f4f38d/users/4206f7f253cd42a29847e169022a2d36/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:53.823480 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd7c25-2986-4f95-baae-028cd38c577d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f71b71b774714529a2b47437e4f4f38d, user_id=4206f7f253cd42a29847e169022a2d36, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:53.826251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd7c25-2986-4f95-baae-028cd38c577d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:53.842144 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1324/2648] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:53 2022] PUT /identity/v3/projects/f71b71b774714529a2b47437e4f4f38d/users/4206f7f253cd42a29847e169022a2d36/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:53.853493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73265700-4cf4-47bd-bf2c-2a003cf82f8c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:53.853809 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73265700-4cf4-47bd-bf2c-2a003cf82f8c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:53.854039 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73265700-4cf4-47bd-bf2c-2a003cf82f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:53.887525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-73265700-4cf4-47bd-bf2c-2a003cf82f8c None None] MFA Rules not processed for user `4206f7f253cd42a29847e169022a2d36`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:53.889373 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-73265700-4cf4-47bd-bf2c-2a003cf82f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:53.999028 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1325/2649] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:54.019191 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd7c25-2986-4f95-baae-028cd38c577d admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:54.114974 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd7c25-2986-4f95-baae-028cd38c577d admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:54.118974 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1ff63-935f-4d0c-b2ce-4d3d38ae9006 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:54.119600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1ff63-935f-4d0c-b2ce-4d3d38ae9006 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:54.119873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1ff63-935f-4d0c-b2ce-4d3d38ae9006 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:54.120140 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1ff63-935f-4d0c-b2ce-4d3d38ae9006 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:54.131531 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1ff63-935f-4d0c-b2ce-4d3d38ae9006 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:54.133181 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1ff63-935f-4d0c-b2ce-4d3d38ae9006 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:54.146281 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1325/2650] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:54.600652 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73265700-4cf4-47bd-bf2c-2a003cf82f8c None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:54.667482 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73265700-4cf4-47bd-bf2c-2a003cf82f8c None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:54.672408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7993b-1457-41f4-beaa-bc586d1845b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:54.673051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7993b-1457-41f4-beaa-bc586d1845b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:54.673351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7993b-1457-41f4-beaa-bc586d1845b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:54.673639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7993b-1457-41f4-beaa-bc586d1845b7 admin admin] PATH_INFO: `/v3/users/916cfaffed5b46f19db6b65acee0552b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:54.674860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7993b-1457-41f4-beaa-bc586d1845b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=916cfaffed5b46f19db6b65acee0552b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:54.678263 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7993b-1457-41f4-beaa-bc586d1845b7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:54.732169 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-16c7993b-1457-41f4-beaa-bc586d1845b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '916cfaffed5b46f19db6b65acee0552b'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:54.740400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-16c7993b-1457-41f4-beaa-bc586d1845b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '916cfaffed5b46f19db6b65acee0552b'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:54.751670 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:54.752930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-16c7993b-1457-41f4-beaa-bc586d1845b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '916cfaffed5b46f19db6b65acee0552b'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:54.754065 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-16c7993b-1457-41f4-beaa-bc586d1845b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '916cfaffed5b46f19db6b65acee0552b'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:54.755560 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-16c7993b-1457-41f4-beaa-bc586d1845b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '916cfaffed5b46f19db6b65acee0552b'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:54.769415 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1326/2651] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:54 2022] DELETE /identity/v3/users/916cfaffed5b46f19db6b65acee0552b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:54.774196 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d55596-17a2-4049-9eb6-4144427fe153 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:54.774478 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d55596-17a2-4049-9eb6-4144427fe153 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:54.774787 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d55596-17a2-4049-9eb6-4144427fe153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:54.781952 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:54.796610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e3d55596-17a2-4049-9eb6-4144427fe153 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:54.797912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e3d55596-17a2-4049-9eb6-4144427fe153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:54.866763 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1326/2652] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:54.877247 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7993b-1457-41f4-beaa-bc586d1845b7 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:54.879021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-16c7993b-1457-41f4-beaa-bc586d1845b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:54.943345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7993b-1457-41f4-beaa-bc586d1845b7 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:54.947422 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c772def-e53c-46a8-861f-f5cf75300796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:54.948042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c772def-e53c-46a8-861f-f5cf75300796 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:54.948325 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c772def-e53c-46a8-861f-f5cf75300796 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:54.948602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c772def-e53c-46a8-861f-f5cf75300796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:54.960211 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c772def-e53c-46a8-861f-f5cf75300796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:54.961758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c772def-e53c-46a8-861f-f5cf75300796 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:54.972193 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1327/2653] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:27:55.288687 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d55596-17a2-4049-9eb6-4144427fe153 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:55.290142 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e3d55596-17a2-4049-9eb6-4144427fe153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:55.304686 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d55596-17a2-4049-9eb6-4144427fe153 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:55.309501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4977c1-f3b9-4f83-ae61-fecb07336a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:55.310143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4977c1-f3b9-4f83-ae61-fecb07336a9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:55.310429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4977c1-f3b9-4f83-ae61-fecb07336a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:55.310703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4977c1-f3b9-4f83-ae61-fecb07336a9b admin admin] PATH_INFO: `/v3/projects/0eeea71a0c344b28b4e7797dea4cc338` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:55.311834 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4977c1-f3b9-4f83-ae61-fecb07336a9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eeea71a0c344b28b4e7797dea4cc338)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:55.314266 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4977c1-f3b9-4f83-ae61-fecb07336a9b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:55.365612 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4b4977c1-f3b9-4f83-ae61-fecb07336a9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eeea71a0c344b28b4e7797dea4cc338', 'request_id': 'req-4b4977c1-f3b9-4f83-ae61-fecb07336a9b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:55.371971 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4b4977c1-f3b9-4f83-ae61-fecb07336a9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eeea71a0c344b28b4e7797dea4cc338', 'request_id': 'req-4b4977c1-f3b9-4f83-ae61-fecb07336a9b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:55.377553 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4b4977c1-f3b9-4f83-ae61-fecb07336a9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eeea71a0c344b28b4e7797dea4cc338', 'request_id': 'req-4b4977c1-f3b9-4f83-ae61-fecb07336a9b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:55.391287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4b4977c1-f3b9-4f83-ae61-fecb07336a9b admin admin] The token cache is being invalidate because project 0eeea71a0c344b28b4e7797dea4cc338 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:55.391583 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4b4977c1-f3b9-4f83-ae61-fecb07336a9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:55.392230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4b4977c1-f3b9-4f83-ae61-fecb07336a9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:55.393780 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1327/2654] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:55 2022] DELETE /identity/v3/projects/0eeea71a0c344b28b4e7797dea4cc338 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:55.398735 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c772def-e53c-46a8-861f-f5cf75300796 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:55.408558 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c772def-e53c-46a8-861f-f5cf75300796 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:55.468103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b5429-d31a-4211-ae40-7a2016e7f517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:55.468742 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b5429-d31a-4211-ae40-7a2016e7f517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:55.469052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b5429-d31a-4211-ae40-7a2016e7f517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:55.469332 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b5429-d31a-4211-ae40-7a2016e7f517 admin admin] PATH_INFO: `/v3/users/4206f7f253cd42a29847e169022a2d36` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:55.470533 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b5429-d31a-4211-ae40-7a2016e7f517 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4206f7f253cd42a29847e169022a2d36)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:55.472510 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b5429-d31a-4211-ae40-7a2016e7f517 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:55.523094 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-742b5429-d31a-4211-ae40-7a2016e7f517 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4206f7f253cd42a29847e169022a2d36'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:55.530425 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-742b5429-d31a-4211-ae40-7a2016e7f517 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4206f7f253cd42a29847e169022a2d36'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:55.542725 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-742b5429-d31a-4211-ae40-7a2016e7f517 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4206f7f253cd42a29847e169022a2d36'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:55.543763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-742b5429-d31a-4211-ae40-7a2016e7f517 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4206f7f253cd42a29847e169022a2d36'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:55.544483 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-742b5429-d31a-4211-ae40-7a2016e7f517 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4206f7f253cd42a29847e169022a2d36'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:55.557394 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1328/2655] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:55 2022] DELETE /identity/v3/users/4206f7f253cd42a29847e169022a2d36 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:55.864503 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4977c1-f3b9-4f83-ae61-fecb07336a9b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:55.866636 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4b4977c1-f3b9-4f83-ae61-fecb07336a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:55.881350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4977c1-f3b9-4f83-ae61-fecb07336a9b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:55.971485 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184939ee-f156-4dfc-8185-2d950cdc52a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:55.972120 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184939ee-f156-4dfc-8185-2d950cdc52a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:55.972420 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184939ee-f156-4dfc-8185-2d950cdc52a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:55.973285 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184939ee-f156-4dfc-8185-2d950cdc52a8 admin admin] PATH_INFO: `/v3/projects/f71b71b774714529a2b47437e4f4f38d` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:55.974544 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184939ee-f156-4dfc-8185-2d950cdc52a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f71b71b774714529a2b47437e4f4f38d)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:55.976479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184939ee-f156-4dfc-8185-2d950cdc52a8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:56.011958 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:56.031186 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-184939ee-f156-4dfc-8185-2d950cdc52a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f71b71b774714529a2b47437e4f4f38d', 'request_id': 'req-184939ee-f156-4dfc-8185-2d950cdc52a8'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:56.037950 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-184939ee-f156-4dfc-8185-2d950cdc52a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f71b71b774714529a2b47437e4f4f38d', 'request_id': 'req-184939ee-f156-4dfc-8185-2d950cdc52a8'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:56.044992 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-184939ee-f156-4dfc-8185-2d950cdc52a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f71b71b774714529a2b47437e4f4f38d', 'request_id': 'req-184939ee-f156-4dfc-8185-2d950cdc52a8'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:56.058866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-184939ee-f156-4dfc-8185-2d950cdc52a8 admin admin] The token cache is being invalidate because project f71b71b774714529a2b47437e4f4f38d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:56.059156 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-184939ee-f156-4dfc-8185-2d950cdc52a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:56.059809 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-184939ee-f156-4dfc-8185-2d950cdc52a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:56.061346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1328/2656] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:55 2022] DELETE /identity/v3/projects/f71b71b774714529a2b47437e4f4f38d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:56.067639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb83ca-361d-4269-9f79-e8b73ac92187 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:56.068227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb83ca-361d-4269-9f79-e8b73ac92187 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:56.068227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb83ca-361d-4269-9f79-e8b73ac92187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:56.096801 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-74cb83ca-361d-4269-9f79-e8b73ac92187 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:56.098524 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-74cb83ca-361d-4269-9f79-e8b73ac92187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:56.104617 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:56.171546 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1329/2657] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:56.179880 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2d405-30b6-4adf-875b-55c71db972d4 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:56.180166 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2d405-30b6-4adf-875b-55c71db972d4 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:56.180443 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2d405-30b6-4adf-875b-55c71db972d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:56.203606 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-9da2d405-30b6-4adf-875b-55c71db972d4 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:56.204978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9da2d405-30b6-4adf-875b-55c71db972d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:56.214970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1329/2658] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:56.219698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb83ca-361d-4269-9f79-e8b73ac92187 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:56.231081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb83ca-361d-4269-9f79-e8b73ac92187 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:56.236751 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789b43a-ac7a-435d-a40d-02cc63e6f63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:56.237526 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789b43a-ac7a-435d-a40d-02cc63e6f63f admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:56.237809 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789b43a-ac7a-435d-a40d-02cc63e6f63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:56.238081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789b43a-ac7a-435d-a40d-02cc63e6f63f admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:56.239077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789b43a-ac7a-435d-a40d-02cc63e6f63f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:27:56.239507 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789b43a-ac7a-435d-a40d-02cc63e6f63f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:56.241187 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789b43a-ac7a-435d-a40d-02cc63e6f63f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:56.256331 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1330/2659] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:27:56.260857 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2d405-30b6-4adf-875b-55c71db972d4 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:56.271571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2d405-30b6-4adf-875b-55c71db972d4 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:56.277386 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed1514-e56c-4d47-a60a-5e4aa38797a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:56.278380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed1514-e56c-4d47-a60a-5e4aa38797a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:56.278835 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed1514-e56c-4d47-a60a-5e4aa38797a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:56.279268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed1514-e56c-4d47-a60a-5e4aa38797a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:56.280118 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed1514-e56c-4d47-a60a-5e4aa38797a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:56.282624 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed1514-e56c-4d47-a60a-5e4aa38797a9 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:56.317274 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1330/2660] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:56 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:56.322953 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789b43a-ac7a-435d-a40d-02cc63e6f63f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:56.343145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789b43a-ac7a-435d-a40d-02cc63e6f63f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:56.428766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e894f-21a4-414d-8394-127b22919095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:56.429582 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e894f-21a4-414d-8394-127b22919095 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:56.429849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e894f-21a4-414d-8394-127b22919095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:56.430177 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e894f-21a4-414d-8394-127b22919095 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:56.430862 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e894f-21a4-414d-8394-127b22919095 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:56.432814 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e894f-21a4-414d-8394-127b22919095 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:56.458215 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1331/2661] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:56 2022] POST /identity/v3/users => generated 283 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:56.462572 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed1514-e56c-4d47-a60a-5e4aa38797a9 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:56.473216 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed1514-e56c-4d47-a60a-5e4aa38797a9 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:56.478411 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d33612-8ee8-43dd-8057-2c3f8a47eaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:56.479054 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d33612-8ee8-43dd-8057-2c3f8a47eaa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:56.479372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d33612-8ee8-43dd-8057-2c3f8a47eaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:56.479810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d33612-8ee8-43dd-8057-2c3f8a47eaa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:56.480357 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d33612-8ee8-43dd-8057-2c3f8a47eaa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:56.481948 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d33612-8ee8-43dd-8057-2c3f8a47eaa6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:56.494202 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1331/2662] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:56.499216 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e894f-21a4-414d-8394-127b22919095 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:56.515200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e894f-21a4-414d-8394-127b22919095 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:56.521097 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953f1ae-6d4e-4487-bded-fb8f5c5b7a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:56.521765 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953f1ae-6d4e-4487-bded-fb8f5c5b7a8e admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:56.522045 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953f1ae-6d4e-4487-bded-fb8f5c5b7a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:56.522333 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953f1ae-6d4e-4487-bded-fb8f5c5b7a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:56.523016 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953f1ae-6d4e-4487-bded-fb8f5c5b7a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:56.537620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953f1ae-6d4e-4487-bded-fb8f5c5b7a8e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:56.537620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1332/2663] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:56.541556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d33612-8ee8-43dd-8057-2c3f8a47eaa6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:56.553862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d33612-8ee8-43dd-8057-2c3f8a47eaa6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:56.559867 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb8382-0edb-4d80-bf56-e8396586e1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:56.560708 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8382-0edb-4d80-bf56-e8396586e1db admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:56.561180 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8382-0edb-4d80-bf56-e8396586e1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:56.561454 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8382-0edb-4d80-bf56-e8396586e1db admin admin] PATH_INFO: `/v3/projects/04d7a3f679a84119a9b403c5f508af56/users/c146966037be4ccbb6bfb74dfa99f894/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:56.579180 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb8382-0edb-4d80-bf56-e8396586e1db admin admin] RBAC: Authorizing `identity:create_grant(project_id=04d7a3f679a84119a9b403c5f508af56, user_id=c146966037be4ccbb6bfb74dfa99f894, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:56.581828 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb8382-0edb-4d80-bf56-e8396586e1db admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:56.603494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1332/2664] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:56 2022] PUT /identity/v3/projects/04d7a3f679a84119a9b403c5f508af56/users/c146966037be4ccbb6bfb74dfa99f894/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:56.627227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bc055-cc3a-4972-b65a-80eeadbdf709 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:56.627575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bc055-cc3a-4972-b65a-80eeadbdf709 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:56.627967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bc055-cc3a-4972-b65a-80eeadbdf709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:56.661829 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9a7bc055-cc3a-4972-b65a-80eeadbdf709 None None] MFA Rules not processed for user `c146966037be4ccbb6bfb74dfa99f894`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:56.663159 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9a7bc055-cc3a-4972-b65a-80eeadbdf709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:56.740494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1333/2665] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:56.745093 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb8382-0edb-4d80-bf56-e8396586e1db admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:56.816307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb8382-0edb-4d80-bf56-e8396586e1db admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:56.821336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb920e79-d255-4cc9-ad64-9ef539e490ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:56.821962 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb920e79-d255-4cc9-ad64-9ef539e490ef admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:56.822252 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb920e79-d255-4cc9-ad64-9ef539e490ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:56.822526 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb920e79-d255-4cc9-ad64-9ef539e490ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:56.823226 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb920e79-d255-4cc9-ad64-9ef539e490ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:56.825189 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb920e79-d255-4cc9-ad64-9ef539e490ef admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:56.856728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1333/2666] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:27:56 2022] POST /identity/v3/projects => generated 363 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:56.860827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bc055-cc3a-4972-b65a-80eeadbdf709 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:56.872935 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bc055-cc3a-4972-b65a-80eeadbdf709 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:56.877010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:56.946984 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf8f08-1f26-4a5a-a22c-442e82473942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:56.947602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf8f08-1f26-4a5a-a22c-442e82473942 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:56.947880 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf8f08-1f26-4a5a-a22c-442e82473942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:56.948148 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf8f08-1f26-4a5a-a22c-442e82473942 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:56.948792 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf8f08-1f26-4a5a-a22c-442e82473942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:56.950722 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf8f08-1f26-4a5a-a22c-442e82473942 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:56.979557 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1334/2667] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:27:56 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:27:56.981208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb920e79-d255-4cc9-ad64-9ef539e490ef admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:56.994405 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb920e79-d255-4cc9-ad64-9ef539e490ef admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:56.999374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b66bea-8d22-473f-bae9-cc9d98ffb3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:56.999992 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b66bea-8d22-473f-bae9-cc9d98ffb3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:57.000275 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b66bea-8d22-473f-bae9-cc9d98ffb3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:57.000548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b66bea-8d22-473f-bae9-cc9d98ffb3f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:57.001214 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b66bea-8d22-473f-bae9-cc9d98ffb3f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:57.002941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b66bea-8d22-473f-bae9-cc9d98ffb3f0 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:57.014802 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1334/2668] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:57.019465 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf8f08-1f26-4a5a-a22c-442e82473942 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:57.031185 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf8f08-1f26-4a5a-a22c-442e82473942 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:57.036604 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb0672-de09-43f6-9dee-efb20bd4e6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:57.037296 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb0672-de09-43f6-9dee-efb20bd4e6ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:57.037573 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb0672-de09-43f6-9dee-efb20bd4e6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:57.037843 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb0672-de09-43f6-9dee-efb20bd4e6ce admin admin] PATH_INFO: `/v3/projects/fc37474cfa974896aa330d6ecf1bb1c6/users/0e60fe234348466b841c1c0af81e016b/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:57.054490 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb0672-de09-43f6-9dee-efb20bd4e6ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc37474cfa974896aa330d6ecf1bb1c6, user_id=0e60fe234348466b841c1c0af81e016b, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:57.057242 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb0672-de09-43f6-9dee-efb20bd4e6ce admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:57.071664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1335/2669] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:57 2022] PUT /identity/v3/projects/fc37474cfa974896aa330d6ecf1bb1c6/users/0e60fe234348466b841c1c0af81e016b/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:57.075796 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b66bea-8d22-473f-bae9-cc9d98ffb3f0 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:57.086517 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b66bea-8d22-473f-bae9-cc9d98ffb3f0 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:57.110965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:57.151030 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01c577-9e9f-4f0f-a202-0200fba86d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:57.151638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01c577-9e9f-4f0f-a202-0200fba86d23 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:57.151918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01c577-9e9f-4f0f-a202-0200fba86d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:57.152192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01c577-9e9f-4f0f-a202-0200fba86d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:57.152845 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01c577-9e9f-4f0f-a202-0200fba86d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:57.154437 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01c577-9e9f-4f0f-a202-0200fba86d23 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:57.165765 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1335/2670] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:27:57.169933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb0672-de09-43f6-9dee-efb20bd4e6ce admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:57.181207 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb0672-de09-43f6-9dee-efb20bd4e6ce admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:57.187260 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050420d-6704-4591-b0da-179ac9396157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:57.187933 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050420d-6704-4591-b0da-179ac9396157 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:57.188212 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050420d-6704-4591-b0da-179ac9396157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:57.188489 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050420d-6704-4591-b0da-179ac9396157 admin admin] PATH_INFO: `/v3/projects/fc37474cfa974896aa330d6ecf1bb1c6/users/0e60fe234348466b841c1c0af81e016b/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:57.190438 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050420d-6704-4591-b0da-179ac9396157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc37474cfa974896aa330d6ecf1bb1c6, user_id=0e60fe234348466b841c1c0af81e016b, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:57.193281 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050420d-6704-4591-b0da-179ac9396157 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:57.204392 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1336/2671] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:27:57 2022] PUT /identity/v3/projects/fc37474cfa974896aa330d6ecf1bb1c6/users/0e60fe234348466b841c1c0af81e016b/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:57.216057 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf2774-480b-4a22-ae2b-2cbf79bd593b None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:57.216371 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf2774-480b-4a22-ae2b-2cbf79bd593b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:57.216645 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf2774-480b-4a22-ae2b-2cbf79bd593b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:57.248792 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-38bf2774-480b-4a22-ae2b-2cbf79bd593b None None] MFA Rules not processed for user `0e60fe234348466b841c1c0af81e016b`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:57.250030 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-38bf2774-480b-4a22-ae2b-2cbf79bd593b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:57.325475 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1336/2672] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:27:57.340400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050420d-6704-4591-b0da-179ac9396157 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:57.342308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4050420d-6704-4591-b0da-179ac9396157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:57.414601 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050420d-6704-4591-b0da-179ac9396157 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:57.420735 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c6d63-0e07-45c1-89b3-db600bb14103 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:57.421806 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c6d63-0e07-45c1-89b3-db600bb14103 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:57.422329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c6d63-0e07-45c1-89b3-db600bb14103 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:57.422877 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c6d63-0e07-45c1-89b3-db600bb14103 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:57.440114 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c6d63-0e07-45c1-89b3-db600bb14103 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:57.443088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c6d63-0e07-45c1-89b3-db600bb14103 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:57.461873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1337/2673] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:27:57.795388 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf2774-480b-4a22-ae2b-2cbf79bd593b None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:57.798002 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-38bf2774-480b-4a22-ae2b-2cbf79bd593b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:57.879547 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf2774-480b-4a22-ae2b-2cbf79bd593b None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:57.884374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a75196-080c-460a-a16a-aec4a895c3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:57.885186 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a75196-080c-460a-a16a-aec4a895c3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:57.885471 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a75196-080c-460a-a16a-aec4a895c3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:57.885747 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a75196-080c-460a-a16a-aec4a895c3ff admin admin] PATH_INFO: `/v3/users/1c4497d3b98541a28331bc8353b25aab` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:57.886997 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a75196-080c-460a-a16a-aec4a895c3ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c4497d3b98541a28331bc8353b25aab)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:57.888949 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a75196-080c-460a-a16a-aec4a895c3ff admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:57.958767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-94a75196-080c-460a-a16a-aec4a895c3ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c4497d3b98541a28331bc8353b25aab'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:57.961696 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:57.969507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-94a75196-080c-460a-a16a-aec4a895c3ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c4497d3b98541a28331bc8353b25aab'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:57.984255 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-94a75196-080c-460a-a16a-aec4a895c3ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c4497d3b98541a28331bc8353b25aab'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:57.985034 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-94a75196-080c-460a-a16a-aec4a895c3ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c4497d3b98541a28331bc8353b25aab'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:57.994644 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-94a75196-080c-460a-a16a-aec4a895c3ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c4497d3b98541a28331bc8353b25aab'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:58.003890 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c6d63-0e07-45c1-89b3-db600bb14103 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:58.008031 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1337/2674] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:57 2022] DELETE /identity/v3/users/1c4497d3b98541a28331bc8353b25aab => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:58.013398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0faee-edaa-4de9-935a-ac4275014853 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:58.013691 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0faee-edaa-4de9-935a-ac4275014853 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:58.013972 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0faee-edaa-4de9-935a-ac4275014853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:58.015606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c6d63-0e07-45c1-89b3-db600bb14103 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:58.036728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d8a0faee-edaa-4de9-935a-ac4275014853 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:58.037770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d8a0faee-edaa-4de9-935a-ac4275014853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:58.091873 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c66f7-ee74-4030-9861-9cd0a1186705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:58.092775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c66f7-ee74-4030-9861-9cd0a1186705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:58.092775 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c66f7-ee74-4030-9861-9cd0a1186705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:58.093134 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c66f7-ee74-4030-9861-9cd0a1186705 admin admin] PATH_INFO: `/v3/users/c146966037be4ccbb6bfb74dfa99f894` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:58.094295 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c66f7-ee74-4030-9861-9cd0a1186705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c146966037be4ccbb6bfb74dfa99f894)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:58.096828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c66f7-ee74-4030-9861-9cd0a1186705 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:58.102084 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:58.119101 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1338/2675] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:27:58.129466 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0faee-edaa-4de9-935a-ac4275014853 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:58.131134 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d8a0faee-edaa-4de9-935a-ac4275014853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:58.167776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fc6c66f7-ee74-4030-9861-9cd0a1186705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c146966037be4ccbb6bfb74dfa99f894'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:58.179774 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fc6c66f7-ee74-4030-9861-9cd0a1186705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c146966037be4ccbb6bfb74dfa99f894'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:58.196654 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fc6c66f7-ee74-4030-9861-9cd0a1186705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c146966037be4ccbb6bfb74dfa99f894'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:58.197878 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fc6c66f7-ee74-4030-9861-9cd0a1186705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c146966037be4ccbb6bfb74dfa99f894'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:58.198816 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fc6c66f7-ee74-4030-9861-9cd0a1186705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c146966037be4ccbb6bfb74dfa99f894'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:58.215711 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1338/2676] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:58 2022] DELETE /identity/v3/users/c146966037be4ccbb6bfb74dfa99f894 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:58.221578 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d27ee6-863c-4fc8-9e3b-4958a3fce53e None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:58.221942 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d27ee6-863c-4fc8-9e3b-4958a3fce53e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:58.222381 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d27ee6-863c-4fc8-9e3b-4958a3fce53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:58.226986 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0faee-edaa-4de9-935a-ac4275014853 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:58.250772 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-74d27ee6-863c-4fc8-9e3b-4958a3fce53e None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:27:58.252013 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-74d27ee6-863c-4fc8-9e3b-4958a3fce53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:58.301767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32260b43-617c-4acd-a80a-4e5d2f453cd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:58.302696 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32260b43-617c-4acd-a80a-4e5d2f453cd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:58.303142 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32260b43-617c-4acd-a80a-4e5d2f453cd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:58.303582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32260b43-617c-4acd-a80a-4e5d2f453cd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:58.306340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-32260b43-617c-4acd-a80a-4e5d2f453cd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:58.339635 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1339/2677] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:27:58.350958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d27ee6-863c-4fc8-9e3b-4958a3fce53e None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:58.352641 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-74d27ee6-863c-4fc8-9e3b-4958a3fce53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:58.366784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d27ee6-863c-4fc8-9e3b-4958a3fce53e None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:58.371711 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6188d-9b61-4085-a46e-60734de7c9dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:58.372345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6188d-9b61-4085-a46e-60734de7c9dd service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:58.372626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6188d-9b61-4085-a46e-60734de7c9dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:58.372897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6188d-9b61-4085-a46e-60734de7c9dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:58.378884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:58.384073 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6188d-9b61-4085-a46e-60734de7c9dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:58.385592 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6188d-9b61-4085-a46e-60734de7c9dd service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:58.398178 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1340/2678] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:27:58.421595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32260b43-617c-4acd-a80a-4e5d2f453cd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:58.423517 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32260b43-617c-4acd-a80a-4e5d2f453cd8 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:58.435955 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1339/2679] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:27:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:27:58.875277 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6188d-9b61-4085-a46e-60734de7c9dd service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:58.877432 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b4c6188d-9b61-4085-a46e-60734de7c9dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:58.900347 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6188d-9b61-4085-a46e-60734de7c9dd service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:58.915886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a2baf-c617-40ab-97d3-03512d55e787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:58.916420 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32260b43-617c-4acd-a80a-4e5d2f453cd8 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:58.917920 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-32260b43-617c-4acd-a80a-4e5d2f453cd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:58.919272 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a2baf-c617-40ab-97d3-03512d55e787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:58.921613 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a2baf-c617-40ab-97d3-03512d55e787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:58.922251 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a2baf-c617-40ab-97d3-03512d55e787 admin admin] PATH_INFO: `/v3/projects/30583c60881b4975a80c79a795e8670b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:58.927402 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a2baf-c617-40ab-97d3-03512d55e787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30583c60881b4975a80c79a795e8670b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:58.930780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a2baf-c617-40ab-97d3-03512d55e787 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:58.934816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32260b43-617c-4acd-a80a-4e5d2f453cd8 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:58.944843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb5a27-485e-440f-9d11-0d650e3af87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:58.945650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb5a27-485e-440f-9d11-0d650e3af87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:58.946981 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb5a27-485e-440f-9d11-0d650e3af87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:58.948826 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb5a27-485e-440f-9d11-0d650e3af87c admin admin] PATH_INFO: `/v3/projects/04d7a3f679a84119a9b403c5f508af56` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:58.950696 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb5a27-485e-440f-9d11-0d650e3af87c admin admin] RBAC: Authorizing `identity:delete_project(project_id=04d7a3f679a84119a9b403c5f508af56)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:58.957642 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb5a27-485e-440f-9d11-0d650e3af87c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:59.022861 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9e6a2baf-c617-40ab-97d3-03512d55e787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30583c60881b4975a80c79a795e8670b', 'request_id': 'req-9e6a2baf-c617-40ab-97d3-03512d55e787'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.034024 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9e6a2baf-c617-40ab-97d3-03512d55e787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30583c60881b4975a80c79a795e8670b', 'request_id': 'req-9e6a2baf-c617-40ab-97d3-03512d55e787'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.038596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-f6fb5a27-485e-440f-9d11-0d650e3af87c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04d7a3f679a84119a9b403c5f508af56', 'request_id': 'req-f6fb5a27-485e-440f-9d11-0d650e3af87c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.042769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:59.045088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9e6a2baf-c617-40ab-97d3-03512d55e787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30583c60881b4975a80c79a795e8670b', 'request_id': 'req-9e6a2baf-c617-40ab-97d3-03512d55e787'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.048442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-f6fb5a27-485e-440f-9d11-0d650e3af87c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04d7a3f679a84119a9b403c5f508af56', 'request_id': 'req-f6fb5a27-485e-440f-9d11-0d650e3af87c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.056858 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-f6fb5a27-485e-440f-9d11-0d650e3af87c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04d7a3f679a84119a9b403c5f508af56', 'request_id': 'req-f6fb5a27-485e-440f-9d11-0d650e3af87c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.063404 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9e6a2baf-c617-40ab-97d3-03512d55e787 admin admin] The token cache is being invalidate because project 30583c60881b4975a80c79a795e8670b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:59.063842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9e6a2baf-c617-40ab-97d3-03512d55e787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.064394 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9e6a2baf-c617-40ab-97d3-03512d55e787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.065963 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1341/2680] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:58 2022] DELETE /identity/v3/projects/30583c60881b4975a80c79a795e8670b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:27:59.071340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a2baf-c617-40ab-97d3-03512d55e787 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:59.072842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9e6a2baf-c617-40ab-97d3-03512d55e787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:59.075247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-f6fb5a27-485e-440f-9d11-0d650e3af87c admin admin] The token cache is being invalidate because project 04d7a3f679a84119a9b403c5f508af56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:59.075703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-f6fb5a27-485e-440f-9d11-0d650e3af87c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.076577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-f6fb5a27-485e-440f-9d11-0d650e3af87c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.078639 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1340/2681] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:58 2022] DELETE /identity/v3/projects/04d7a3f679a84119a9b403c5f508af56 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:27:59.084469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb5a27-485e-440f-9d11-0d650e3af87c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:59.086322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f6fb5a27-485e-440f-9d11-0d650e3af87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:59.151552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a2baf-c617-40ab-97d3-03512d55e787 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:59.189297 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb5a27-485e-440f-9d11-0d650e3af87c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:59.195119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4c9bb-1287-40f3-9738-4152658aac50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:59.195756 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4c9bb-1287-40f3-9738-4152658aac50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:59.196033 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4c9bb-1287-40f3-9738-4152658aac50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:59.196304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4c9bb-1287-40f3-9738-4152658aac50 admin admin] PATH_INFO: `/v3/users/0e60fe234348466b841c1c0af81e016b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:59.197490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4c9bb-1287-40f3-9738-4152658aac50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e60fe234348466b841c1c0af81e016b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:59.199472 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4c9bb-1287-40f3-9738-4152658aac50 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:59.231257 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb4f76-d9d2-4e66-8d6c-bb1c4737908b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:59.231909 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb4f76-d9d2-4e66-8d6c-bb1c4737908b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:59.232193 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb4f76-d9d2-4e66-8d6c-bb1c4737908b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:59.232489 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb4f76-d9d2-4e66-8d6c-bb1c4737908b admin admin] PATH_INFO: `/v3/users/1c5b1e9a6d2c4eb3928f137df7fc7193` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:59.233699 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb4f76-d9d2-4e66-8d6c-bb1c4737908b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c5b1e9a6d2c4eb3928f137df7fc7193)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:59.235989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb4f76-d9d2-4e66-8d6c-bb1c4737908b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:59.272488 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-84a4c9bb-1287-40f3-9738-4152658aac50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e60fe234348466b841c1c0af81e016b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.287199 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-84a4c9bb-1287-40f3-9738-4152658aac50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e60fe234348466b841c1c0af81e016b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.291339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:59.309578 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-84a4c9bb-1287-40f3-9738-4152658aac50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e60fe234348466b841c1c0af81e016b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.310362 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-84a4c9bb-1287-40f3-9738-4152658aac50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e60fe234348466b841c1c0af81e016b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.316887 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cdbb4f76-d9d2-4e66-8d6c-bb1c4737908b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c5b1e9a6d2c4eb3928f137df7fc7193'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.320372 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-84a4c9bb-1287-40f3-9738-4152658aac50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e60fe234348466b841c1c0af81e016b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.326386 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cdbb4f76-d9d2-4e66-8d6c-bb1c4737908b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c5b1e9a6d2c4eb3928f137df7fc7193'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.328200 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1341/2682] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:59 2022] DELETE /identity/v3/users/0e60fe234348466b841c1c0af81e016b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:27:59.341352 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cdbb4f76-d9d2-4e66-8d6c-bb1c4737908b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c5b1e9a6d2c4eb3928f137df7fc7193'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.342126 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cdbb4f76-d9d2-4e66-8d6c-bb1c4737908b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c5b1e9a6d2c4eb3928f137df7fc7193'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.342816 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cdbb4f76-d9d2-4e66-8d6c-bb1c4737908b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c5b1e9a6d2c4eb3928f137df7fc7193'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.358110 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1342/2683] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:59 2022] DELETE /identity/v3/users/1c5b1e9a6d2c4eb3928f137df7fc7193 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:27:59.758681 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4c9bb-1287-40f3-9738-4152658aac50 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:59.762727 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-84a4c9bb-1287-40f3-9738-4152658aac50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:59.781073 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb4f76-d9d2-4e66-8d6c-bb1c4737908b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:59.783096 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cdbb4f76-d9d2-4e66-8d6c-bb1c4737908b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:59.815186 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:27:59.870441 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb4f76-d9d2-4e66-8d6c-bb1c4737908b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:59.876265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677731d-d14b-43e9-bb6f-7246257e0c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:59.876956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677731d-d14b-43e9-bb6f-7246257e0c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:59.877190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677731d-d14b-43e9-bb6f-7246257e0c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:59.877470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677731d-d14b-43e9-bb6f-7246257e0c2f admin admin] PATH_INFO: `/v3/projects/f7705615c50244f1a1ae29e38afe5a7e` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:59.878627 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677731d-d14b-43e9-bb6f-7246257e0c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7705615c50244f1a1ae29e38afe5a7e)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:59.882000 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677731d-d14b-43e9-bb6f-7246257e0c2f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:59.890874 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4c9bb-1287-40f3-9738-4152658aac50 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:27:59.897620 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027a91b-71ed-4e4a-bb12-ba8b4f0e7c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:27:59.898461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027a91b-71ed-4e4a-bb12-ba8b4f0e7c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:27:59.898818 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027a91b-71ed-4e4a-bb12-ba8b4f0e7c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:27:59.899136 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027a91b-71ed-4e4a-bb12-ba8b4f0e7c9c admin admin] PATH_INFO: `/v3/projects/fc37474cfa974896aa330d6ecf1bb1c6` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:27:59.900623 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027a91b-71ed-4e4a-bb12-ba8b4f0e7c9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc37474cfa974896aa330d6ecf1bb1c6)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:27:59.903063 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027a91b-71ed-4e4a-bb12-ba8b4f0e7c9c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:27:59.943420 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c677731d-d14b-43e9-bb6f-7246257e0c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7705615c50244f1a1ae29e38afe5a7e', 'request_id': 'req-c677731d-d14b-43e9-bb6f-7246257e0c2f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.952748 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c677731d-d14b-43e9-bb6f-7246257e0c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7705615c50244f1a1ae29e38afe5a7e', 'request_id': 'req-c677731d-d14b-43e9-bb6f-7246257e0c2f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.958009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0027a91b-71ed-4e4a-bb12-ba8b4f0e7c9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc37474cfa974896aa330d6ecf1bb1c6', 'request_id': 'req-0027a91b-71ed-4e4a-bb12-ba8b4f0e7c9c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.959779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c677731d-d14b-43e9-bb6f-7246257e0c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7705615c50244f1a1ae29e38afe5a7e', 'request_id': 'req-c677731d-d14b-43e9-bb6f-7246257e0c2f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.964874 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0027a91b-71ed-4e4a-bb12-ba8b4f0e7c9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc37474cfa974896aa330d6ecf1bb1c6', 'request_id': 'req-0027a91b-71ed-4e4a-bb12-ba8b4f0e7c9c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.971162 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0027a91b-71ed-4e4a-bb12-ba8b4f0e7c9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc37474cfa974896aa330d6ecf1bb1c6', 'request_id': 'req-0027a91b-71ed-4e4a-bb12-ba8b4f0e7c9c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.973277 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c677731d-d14b-43e9-bb6f-7246257e0c2f admin admin] The token cache is being invalidate because project f7705615c50244f1a1ae29e38afe5a7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:59.973628 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c677731d-d14b-43e9-bb6f-7246257e0c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.974599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c677731d-d14b-43e9-bb6f-7246257e0c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.977065 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1343/2684] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:59 2022] DELETE /identity/v3/projects/f7705615c50244f1a1ae29e38afe5a7e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:27:59.981698 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677731d-d14b-43e9-bb6f-7246257e0c2f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:27:59.983833 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c677731d-d14b-43e9-bb6f-7246257e0c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:27:59.993423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0027a91b-71ed-4e4a-bb12-ba8b4f0e7c9c admin admin] The token cache is being invalidate because project fc37474cfa974896aa330d6ecf1bb1c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:27:59.993685 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0027a91b-71ed-4e4a-bb12-ba8b4f0e7c9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.995088 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0027a91b-71ed-4e4a-bb12-ba8b4f0e7c9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:27:59.996914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1342/2685] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:27:59 2022] DELETE /identity/v3/projects/fc37474cfa974896aa330d6ecf1bb1c6 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:28:00.004629 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bc794-0430-40ce-a0a2-fe9c38a49b57 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:00.004970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bc794-0430-40ce-a0a2-fe9c38a49b57 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:00.005248 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bc794-0430-40ce-a0a2-fe9c38a49b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:00.027625 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6c4bc794-0430-40ce-a0a2-fe9c38a49b57 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:00.028985 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6c4bc794-0430-40ce-a0a2-fe9c38a49b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:00.063606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677731d-d14b-43e9-bb6f-7246257e0c2f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:00.105827 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1343/2686] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:00.107611 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe3376-d730-4696-9c59-d5d928b1a341 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:00.107907 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe3376-d730-4696-9c59-d5d928b1a341 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:00.108211 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe3376-d730-4696-9c59-d5d928b1a341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:00.126158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:00.133104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:00.141088 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-cffe3376-d730-4696-9c59-d5d928b1a341 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:00.143080 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a505eb-751c-44bb-af7b-0127faf8fc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:00.143455 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cffe3376-d730-4696-9c59-d5d928b1a341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:00.144228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a505eb-751c-44bb-af7b-0127faf8fc5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:00.145155 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a505eb-751c-44bb-af7b-0127faf8fc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:00.145773 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a505eb-751c-44bb-af7b-0127faf8fc5f admin admin] PATH_INFO: `/v3/users/3c00ac1788af4298a82c141455e2c7f4` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:00.147305 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a505eb-751c-44bb-af7b-0127faf8fc5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c00ac1788af4298a82c141455e2c7f4)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:00.149339 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a505eb-751c-44bb-af7b-0127faf8fc5f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:00.153511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1344/2687] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:00.157795 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe3376-d730-4696-9c59-d5d928b1a341 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:00.168937 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe3376-d730-4696-9c59-d5d928b1a341 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:00.173904 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392b656-628c-4088-8539-2531562dfa2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:00.174512 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392b656-628c-4088-8539-2531562dfa2b admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:00.174816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392b656-628c-4088-8539-2531562dfa2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:00.175090 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392b656-628c-4088-8539-2531562dfa2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:00.175834 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392b656-628c-4088-8539-2531562dfa2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:28:00.176465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392b656-628c-4088-8539-2531562dfa2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:00.178072 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392b656-628c-4088-8539-2531562dfa2b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:00.192126 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1345/2688] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:28:00.196805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392b656-628c-4088-8539-2531562dfa2b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:00.205375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-97a505eb-751c-44bb-af7b-0127faf8fc5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c00ac1788af4298a82c141455e2c7f4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:00.208719 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392b656-628c-4088-8539-2531562dfa2b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:00.213061 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-97a505eb-751c-44bb-af7b-0127faf8fc5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c00ac1788af4298a82c141455e2c7f4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:00.213911 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b553ed4-bfd9-4b39-bb28-795e6c086017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:00.214522 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b553ed4-bfd9-4b39-bb28-795e6c086017 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:00.214850 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b553ed4-bfd9-4b39-bb28-795e6c086017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:00.215123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b553ed4-bfd9-4b39-bb28-795e6c086017 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:00.215763 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b553ed4-bfd9-4b39-bb28-795e6c086017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:00.217718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b553ed4-bfd9-4b39-bb28-795e6c086017 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:00.232310 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-97a505eb-751c-44bb-af7b-0127faf8fc5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c00ac1788af4298a82c141455e2c7f4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:00.233302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-97a505eb-751c-44bb-af7b-0127faf8fc5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c00ac1788af4298a82c141455e2c7f4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:00.233892 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-97a505eb-751c-44bb-af7b-0127faf8fc5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c00ac1788af4298a82c141455e2c7f4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:00.258638 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1344/2689] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:27:59 2022] DELETE /identity/v3/users/3c00ac1788af4298a82c141455e2c7f4 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:28:00.258904 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1346/2690] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:28:00 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:00.259759 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a505eb-751c-44bb-af7b-0127faf8fc5f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:00.262203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-97a505eb-751c-44bb-af7b-0127faf8fc5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:00.352327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a505eb-751c-44bb-af7b-0127faf8fc5f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:00.452469 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62577f20-8dbb-4f4d-8d3b-28094abed836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:00.452763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62577f20-8dbb-4f4d-8d3b-28094abed836 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:00.453192 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62577f20-8dbb-4f4d-8d3b-28094abed836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:00.453482 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62577f20-8dbb-4f4d-8d3b-28094abed836 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:00.454257 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62577f20-8dbb-4f4d-8d3b-28094abed836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:00.456234 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62577f20-8dbb-4f4d-8d3b-28094abed836 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:00.493763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1345/2691] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:28:00 2022] POST /identity/v3/users => generated 287 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:28:00.495051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b553ed4-bfd9-4b39-bb28-795e6c086017 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:00.506554 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b553ed4-bfd9-4b39-bb28-795e6c086017 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:00.511588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a166b1-3dc4-488e-9f64-9148d74e335d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:00.512197 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a166b1-3dc4-488e-9f64-9148d74e335d admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:00.512480 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a166b1-3dc4-488e-9f64-9148d74e335d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:00.512750 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a166b1-3dc4-488e-9f64-9148d74e335d admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:00.513396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a166b1-3dc4-488e-9f64-9148d74e335d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:00.515060 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a166b1-3dc4-488e-9f64-9148d74e335d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:00.527287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1347/2692] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:00.531796 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62577f20-8dbb-4f4d-8d3b-28094abed836 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:00.543599 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62577f20-8dbb-4f4d-8d3b-28094abed836 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:00.548937 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5caddb-0919-4f00-a380-24761dcfdf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:00.549561 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5caddb-0919-4f00-a380-24761dcfdf94 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:00.549874 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5caddb-0919-4f00-a380-24761dcfdf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:00.550177 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5caddb-0919-4f00-a380-24761dcfdf94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:00.551562 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5caddb-0919-4f00-a380-24761dcfdf94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:00.554920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5caddb-0919-4f00-a380-24761dcfdf94 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:00.563505 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:00.575714 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1346/2693] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:28:00.577298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a166b1-3dc4-488e-9f64-9148d74e335d admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:00.592212 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5caddb-0919-4f00-a380-24761dcfdf94 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:00.594551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6a5caddb-0919-4f00-a380-24761dcfdf94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:00.600932 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a166b1-3dc4-488e-9f64-9148d74e335d admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:00.612670 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b2b69-63c2-4a81-ba75-a806b45d60c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:00.614442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b2b69-63c2-4a81-ba75-a806b45d60c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:00.615058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b2b69-63c2-4a81-ba75-a806b45d60c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:00.616279 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b2b69-63c2-4a81-ba75-a806b45d60c8 admin admin] PATH_INFO: `/v3/projects/eb2500d33f704e80b7e5b5b38413e535/users/8842f7ebf9664eb9aa299287e1fcd36e/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:00.620731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5caddb-0919-4f00-a380-24761dcfdf94 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:00.626602 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc1171-46d7-4998-ae2e-74ed410c9b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:00.627305 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc1171-46d7-4998-ae2e-74ed410c9b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:00.627623 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc1171-46d7-4998-ae2e-74ed410c9b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:00.628041 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc1171-46d7-4998-ae2e-74ed410c9b71 admin admin] PATH_INFO: `/v3/projects/96718d133e5047bab511086a48bc1d31` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:00.630065 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc1171-46d7-4998-ae2e-74ed410c9b71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96718d133e5047bab511086a48bc1d31)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:00.632109 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc1171-46d7-4998-ae2e-74ed410c9b71 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:00.646327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b2b69-63c2-4a81-ba75-a806b45d60c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb2500d33f704e80b7e5b5b38413e535, user_id=8842f7ebf9664eb9aa299287e1fcd36e, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:00.651700 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b2b69-63c2-4a81-ba75-a806b45d60c8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:00.673003 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1348/2694] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:28:00 2022] PUT /identity/v3/projects/eb2500d33f704e80b7e5b5b38413e535/users/8842f7ebf9664eb9aa299287e1fcd36e/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:00.679698 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5cb5b-470f-4582-aa63-6e1f6010bdbc None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:00.680111 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5cb5b-470f-4582-aa63-6e1f6010bdbc None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:00.680513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5cb5b-470f-4582-aa63-6e1f6010bdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:00.698565 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d9fc1171-46d7-4998-ae2e-74ed410c9b71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96718d133e5047bab511086a48bc1d31', 'request_id': 'req-d9fc1171-46d7-4998-ae2e-74ed410c9b71'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:00.706335 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-40c5cb5b-470f-4582-aa63-6e1f6010bdbc None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:00.708389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-40c5cb5b-470f-4582-aa63-6e1f6010bdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:00.709754 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d9fc1171-46d7-4998-ae2e-74ed410c9b71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96718d133e5047bab511086a48bc1d31', 'request_id': 'req-d9fc1171-46d7-4998-ae2e-74ed410c9b71'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:00.718860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d9fc1171-46d7-4998-ae2e-74ed410c9b71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96718d133e5047bab511086a48bc1d31', 'request_id': 'req-d9fc1171-46d7-4998-ae2e-74ed410c9b71'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:00.735528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d9fc1171-46d7-4998-ae2e-74ed410c9b71 admin admin] The token cache is being invalidate because project 96718d133e5047bab511086a48bc1d31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:28:00.736001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d9fc1171-46d7-4998-ae2e-74ed410c9b71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:00.736889 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d9fc1171-46d7-4998-ae2e-74ed410c9b71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:00.739758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1347/2695] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:28:00 2022] DELETE /identity/v3/projects/96718d133e5047bab511086a48bc1d31 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:00.783466 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1349/2696] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:28:00.795223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc1171-46d7-4998-ae2e-74ed410c9b71 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:00.796800 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d9fc1171-46d7-4998-ae2e-74ed410c9b71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:00.873517 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc1171-46d7-4998-ae2e-74ed410c9b71 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:00.878138 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac662e92-3662-4611-8ee1-19c69f9971ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:00.879300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac662e92-3662-4611-8ee1-19c69f9971ca service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:00.879588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac662e92-3662-4611-8ee1-19c69f9971ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:00.879899 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac662e92-3662-4611-8ee1-19c69f9971ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:00.891485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac662e92-3662-4611-8ee1-19c69f9971ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:00.893025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac662e92-3662-4611-8ee1-19c69f9971ca service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:00.905297 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1348/2697] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:01.599586 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d65b82-c208-4b7e-9784-fb7f9ef164e2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:01.599882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d65b82-c208-4b7e-9784-fb7f9ef164e2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:01.600766 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d65b82-c208-4b7e-9784-fb7f9ef164e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:01.634394 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-75d65b82-c208-4b7e-9784-fb7f9ef164e2 None None] MFA Rules not processed for user `8842f7ebf9664eb9aa299287e1fcd36e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:01.635860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-75d65b82-c208-4b7e-9784-fb7f9ef164e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:01.716567 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1350/2698] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:01.721056 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac662e92-3662-4611-8ee1-19c69f9971ca service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:01.722683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ac662e92-3662-4611-8ee1-19c69f9971ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:01.736594 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac662e92-3662-4611-8ee1-19c69f9971ca service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:01.741578 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e15e22-e866-4fc4-91a3-f7894fbb58c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:01.742282 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e15e22-e866-4fc4-91a3-f7894fbb58c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:01.742635 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e15e22-e866-4fc4-91a3-f7894fbb58c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:01.742966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e15e22-e866-4fc4-91a3-f7894fbb58c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:01.743616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e15e22-e866-4fc4-91a3-f7894fbb58c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:01.745534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e15e22-e866-4fc4-91a3-f7894fbb58c8 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:01.777978 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1349/2699] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:28:01 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:01.782481 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d65b82-c208-4b7e-9784-fb7f9ef164e2 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:01.794483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d65b82-c208-4b7e-9784-fb7f9ef164e2 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:01.812815 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:01.860309 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fb500-ee9c-41b5-bf72-0138fab220f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:01.860924 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fb500-ee9c-41b5-bf72-0138fab220f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:01.861205 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fb500-ee9c-41b5-bf72-0138fab220f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:01.861479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fb500-ee9c-41b5-bf72-0138fab220f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:01.862143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848fb500-ee9c-41b5-bf72-0138fab220f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:01.864152 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848fb500-ee9c-41b5-bf72-0138fab220f8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:01.887777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1351/2700] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:28:01 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:01.891984 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e15e22-e866-4fc4-91a3-f7894fbb58c8 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:01.903762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e15e22-e866-4fc4-91a3-f7894fbb58c8 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:01.908648 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d33b69-b212-4e9f-8169-8e15443e2af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:01.909262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d33b69-b212-4e9f-8169-8e15443e2af8 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:01.913995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d33b69-b212-4e9f-8169-8e15443e2af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:01.913995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d33b69-b212-4e9f-8169-8e15443e2af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:01.913995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d33b69-b212-4e9f-8169-8e15443e2af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:01.913995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d33b69-b212-4e9f-8169-8e15443e2af8 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:01.924317 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1350/2701] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:01.929106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fb500-ee9c-41b5-bf72-0138fab220f8 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:01.939830 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fb500-ee9c-41b5-bf72-0138fab220f8 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:01.944743 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035a58e-1e16-40c8-a823-adb0680a7644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:01.945407 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035a58e-1e16-40c8-a823-adb0680a7644 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:01.945689 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035a58e-1e16-40c8-a823-adb0680a7644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:01.945965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035a58e-1e16-40c8-a823-adb0680a7644 admin admin] PATH_INFO: `/v3/projects/4974b4eb7a114512996d4eb8250df2f1/users/9b2811326559428795240c0a4f685ad6/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:01.963296 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035a58e-1e16-40c8-a823-adb0680a7644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4974b4eb7a114512996d4eb8250df2f1, user_id=9b2811326559428795240c0a4f685ad6, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:01.965995 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035a58e-1e16-40c8-a823-adb0680a7644 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:01.981841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1352/2702] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:28:01 2022] PUT /identity/v3/projects/4974b4eb7a114512996d4eb8250df2f1/users/9b2811326559428795240c0a4f685ad6/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:28:01.986547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d33b69-b212-4e9f-8169-8e15443e2af8 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:01.996985 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d33b69-b212-4e9f-8169-8e15443e2af8 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:02.045204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:02.057499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3257e0-193a-4fc6-85a6-3729c41cd876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:02.058137 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3257e0-193a-4fc6-85a6-3729c41cd876 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:02.058410 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3257e0-193a-4fc6-85a6-3729c41cd876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:02.058709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3257e0-193a-4fc6-85a6-3729c41cd876 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:02.059511 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3257e0-193a-4fc6-85a6-3729c41cd876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:02.061148 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3257e0-193a-4fc6-85a6-3729c41cd876 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:02.072257 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1351/2703] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:02.076542 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035a58e-1e16-40c8-a823-adb0680a7644 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:02.088099 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035a58e-1e16-40c8-a823-adb0680a7644 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:02.093274 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5047b-2d98-4ff9-aa21-eb204b75a868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:02.093988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5047b-2d98-4ff9-aa21-eb204b75a868 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:02.094267 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5047b-2d98-4ff9-aa21-eb204b75a868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:02.094533 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5047b-2d98-4ff9-aa21-eb204b75a868 admin admin] PATH_INFO: `/v3/projects/4974b4eb7a114512996d4eb8250df2f1/users/9b2811326559428795240c0a4f685ad6/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:02.098475 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5047b-2d98-4ff9-aa21-eb204b75a868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4974b4eb7a114512996d4eb8250df2f1, user_id=9b2811326559428795240c0a4f685ad6, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:02.100511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5047b-2d98-4ff9-aa21-eb204b75a868 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:02.111643 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1353/2704] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:28:02 2022] PUT /identity/v3/projects/4974b4eb7a114512996d4eb8250df2f1/users/9b2811326559428795240c0a4f685ad6/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:02.780799 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d93ae9-4176-4902-a187-c6112cbf8bd7 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:02.781356 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d93ae9-4176-4902-a187-c6112cbf8bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:02.781356 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d93ae9-4176-4902-a187-c6112cbf8bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:02.810509 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-13d93ae9-4176-4902-a187-c6112cbf8bd7 None None] MFA Rules not processed for user `9b2811326559428795240c0a4f685ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:02.811732 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-13d93ae9-4176-4902-a187-c6112cbf8bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:02.887311 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1352/2705] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:02.899614 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5047b-2d98-4ff9-aa21-eb204b75a868 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:02.901342 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-85a5047b-2d98-4ff9-aa21-eb204b75a868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:02.967707 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5047b-2d98-4ff9-aa21-eb204b75a868 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:02.971352 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c64c22-b97e-46e3-9bcd-165151833ac5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:02.971949 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c64c22-b97e-46e3-9bcd-165151833ac5 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:02.972228 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c64c22-b97e-46e3-9bcd-165151833ac5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:02.972520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c64c22-b97e-46e3-9bcd-165151833ac5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:03.031965 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c64c22-b97e-46e3-9bcd-165151833ac5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:03.033405 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c64c22-b97e-46e3-9bcd-165151833ac5 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:03.043666 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1354/2706] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:04.419685 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d93ae9-4176-4902-a187-c6112cbf8bd7 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:04.421085 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-13d93ae9-4176-4902-a187-c6112cbf8bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:04.433362 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d93ae9-4176-4902-a187-c6112cbf8bd7 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:04.437167 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7f9bd-4dc4-4fc5-9211-5f11a12b7eef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:04.437756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7f9bd-4dc4-4fc5-9211-5f11a12b7eef service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:04.438026 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7f9bd-4dc4-4fc5-9211-5f11a12b7eef service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:04.438571 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7f9bd-4dc4-4fc5-9211-5f11a12b7eef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:04.448341 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7f9bd-4dc4-4fc5-9211-5f11a12b7eef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:04.449779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7f9bd-4dc4-4fc5-9211-5f11a12b7eef service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:04.460088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1353/2707] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:28:13.038710 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=83,keystone:INSERT=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:14.455110 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=45 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:17.618141 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c64c22-b97e-46e3-9bcd-165151833ac5 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:17.683808 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c64c22-b97e-46e3-9bcd-165151833ac5 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:17.688640 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd41bc-b8e4-419c-ac1e-95000b58b544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:17.689259 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd41bc-b8e4-419c-ac1e-95000b58b544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:17.689536 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd41bc-b8e4-419c-ac1e-95000b58b544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:17.689806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd41bc-b8e4-419c-ac1e-95000b58b544 admin admin] PATH_INFO: `/v3/users/8842f7ebf9664eb9aa299287e1fcd36e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:17.690952 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd41bc-b8e4-419c-ac1e-95000b58b544 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8842f7ebf9664eb9aa299287e1fcd36e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:17.693496 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd41bc-b8e4-419c-ac1e-95000b58b544 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:17.739931 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cdfd41bc-b8e4-419c-ac1e-95000b58b544 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8842f7ebf9664eb9aa299287e1fcd36e'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:17.746927 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cdfd41bc-b8e4-419c-ac1e-95000b58b544 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8842f7ebf9664eb9aa299287e1fcd36e'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:17.759125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cdfd41bc-b8e4-419c-ac1e-95000b58b544 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8842f7ebf9664eb9aa299287e1fcd36e'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:17.759821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cdfd41bc-b8e4-419c-ac1e-95000b58b544 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8842f7ebf9664eb9aa299287e1fcd36e'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:17.767378 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cdfd41bc-b8e4-419c-ac1e-95000b58b544 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8842f7ebf9664eb9aa299287e1fcd36e'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:17.774057 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1355/2708] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:28:17 2022] DELETE /identity/v3/users/8842f7ebf9664eb9aa299287e1fcd36e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:17.946448 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7f9bd-4dc4-4fc5-9211-5f11a12b7eef service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:17.965682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7f9bd-4dc4-4fc5-9211-5f11a12b7eef service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:18.031810 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed908f-6737-41e6-a158-96cfcdb4ba69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:18.032438 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed908f-6737-41e6-a158-96cfcdb4ba69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:18.032721 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed908f-6737-41e6-a158-96cfcdb4ba69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:18.033004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed908f-6737-41e6-a158-96cfcdb4ba69 admin admin] PATH_INFO: `/v3/projects/eb2500d33f704e80b7e5b5b38413e535` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:18.034125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed908f-6737-41e6-a158-96cfcdb4ba69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb2500d33f704e80b7e5b5b38413e535)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:18.036331 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed908f-6737-41e6-a158-96cfcdb4ba69 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:18.082507 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a4ed908f-6737-41e6-a158-96cfcdb4ba69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb2500d33f704e80b7e5b5b38413e535', 'request_id': 'req-a4ed908f-6737-41e6-a158-96cfcdb4ba69'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:18.090068 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a4ed908f-6737-41e6-a158-96cfcdb4ba69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb2500d33f704e80b7e5b5b38413e535', 'request_id': 'req-a4ed908f-6737-41e6-a158-96cfcdb4ba69'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:18.095749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a4ed908f-6737-41e6-a158-96cfcdb4ba69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb2500d33f704e80b7e5b5b38413e535', 'request_id': 'req-a4ed908f-6737-41e6-a158-96cfcdb4ba69'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:18.106732 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a4ed908f-6737-41e6-a158-96cfcdb4ba69 admin admin] The token cache is being invalidate because project eb2500d33f704e80b7e5b5b38413e535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:28:18.107030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a4ed908f-6737-41e6-a158-96cfcdb4ba69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:18.107673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a4ed908f-6737-41e6-a158-96cfcdb4ba69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:18.109177 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1354/2709] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:28:17 2022] DELETE /identity/v3/projects/eb2500d33f704e80b7e5b5b38413e535 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:18.114582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd41bc-b8e4-419c-ac1e-95000b58b544 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:18.116484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cdfd41bc-b8e4-419c-ac1e-95000b58b544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:18.130612 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd41bc-b8e4-419c-ac1e-95000b58b544 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:18.188679 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2cd4a-e401-4513-bf4b-11562352bd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:18.189572 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2cd4a-e401-4513-bf4b-11562352bd2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:18.189858 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2cd4a-e401-4513-bf4b-11562352bd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:18.190133 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2cd4a-e401-4513-bf4b-11562352bd2f admin admin] PATH_INFO: `/v3/users/9b2811326559428795240c0a4f685ad6` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:18.192014 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2cd4a-e401-4513-bf4b-11562352bd2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b2811326559428795240c0a4f685ad6)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:18.193976 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2cd4a-e401-4513-bf4b-11562352bd2f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:18.215626 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:18.243644 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a5c2cd4a-e401-4513-bf4b-11562352bd2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b2811326559428795240c0a4f685ad6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:18.250775 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a5c2cd4a-e401-4513-bf4b-11562352bd2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b2811326559428795240c0a4f685ad6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:18.263402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a5c2cd4a-e401-4513-bf4b-11562352bd2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b2811326559428795240c0a4f685ad6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:18.264143 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a5c2cd4a-e401-4513-bf4b-11562352bd2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b2811326559428795240c0a4f685ad6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:18.271089 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a5c2cd4a-e401-4513-bf4b-11562352bd2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b2811326559428795240c0a4f685ad6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:18.277171 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1356/2710] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:28:18 2022] DELETE /identity/v3/users/9b2811326559428795240c0a4f685ad6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:18.470259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed908f-6737-41e6-a158-96cfcdb4ba69 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:18.471754 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a4ed908f-6737-41e6-a158-96cfcdb4ba69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:18.485794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed908f-6737-41e6-a158-96cfcdb4ba69 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:18.545051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43194493-50c8-4e36-a004-8081efe53f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:18.545687 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43194493-50c8-4e36-a004-8081efe53f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:18.545967 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43194493-50c8-4e36-a004-8081efe53f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:18.546246 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43194493-50c8-4e36-a004-8081efe53f34 admin admin] PATH_INFO: `/v3/projects/4974b4eb7a114512996d4eb8250df2f1` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:18.547379 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43194493-50c8-4e36-a004-8081efe53f34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4974b4eb7a114512996d4eb8250df2f1)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:18.549281 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43194493-50c8-4e36-a004-8081efe53f34 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:18.590120 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:18.608158 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-43194493-50c8-4e36-a004-8081efe53f34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4974b4eb7a114512996d4eb8250df2f1', 'request_id': 'req-43194493-50c8-4e36-a004-8081efe53f34'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:18.616563 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-43194493-50c8-4e36-a004-8081efe53f34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4974b4eb7a114512996d4eb8250df2f1', 'request_id': 'req-43194493-50c8-4e36-a004-8081efe53f34'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:18.622276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-43194493-50c8-4e36-a004-8081efe53f34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4974b4eb7a114512996d4eb8250df2f1', 'request_id': 'req-43194493-50c8-4e36-a004-8081efe53f34'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:18.634067 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-43194493-50c8-4e36-a004-8081efe53f34 admin admin] The token cache is being invalidate because project 4974b4eb7a114512996d4eb8250df2f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:28:18.634357 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-43194493-50c8-4e36-a004-8081efe53f34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:18.634998 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-43194493-50c8-4e36-a004-8081efe53f34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:18.636672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1355/2711] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:28:18 2022] DELETE /identity/v3/projects/4974b4eb7a114512996d4eb8250df2f1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:18.641972 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb4650-09e6-43cc-9b05-254723317db0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:18.642453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb4650-09e6-43cc-9b05-254723317db0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:18.642453 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb4650-09e6-43cc-9b05-254723317db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:18.663699 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e7cb4650-09e6-43cc-9b05-254723317db0 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:18.664915 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e7cb4650-09e6-43cc-9b05-254723317db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:18.724609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1357/2712] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:18.733835 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2a8ca-f6ee-472b-8f83-50f27697bb90 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:18.734119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2a8ca-f6ee-472b-8f83-50f27697bb90 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:18.734425 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2a8ca-f6ee-472b-8f83-50f27697bb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:18.755132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-c6e2a8ca-f6ee-472b-8f83-50f27697bb90 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:18.756353 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c6e2a8ca-f6ee-472b-8f83-50f27697bb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:18.764557 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1356/2713] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:18.769600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb4650-09e6-43cc-9b05-254723317db0 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:18.779768 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb4650-09e6-43cc-9b05-254723317db0 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:18.784505 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a680e-e9ae-432a-9feb-5761032d06dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:18.785600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a680e-e9ae-432a-9feb-5761032d06dd admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:18.785886 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a680e-e9ae-432a-9feb-5761032d06dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:18.786166 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a680e-e9ae-432a-9feb-5761032d06dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:18.786844 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a680e-e9ae-432a-9feb-5761032d06dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:28:18.787147 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a680e-e9ae-432a-9feb-5761032d06dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:18.788742 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a680e-e9ae-432a-9feb-5761032d06dd admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:18.802489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1358/2714] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:28:18.807135 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2a8ca-f6ee-472b-8f83-50f27697bb90 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:18.816538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2a8ca-f6ee-472b-8f83-50f27697bb90 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:18.821436 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edaaa8c-dae1-4741-aa37-87553151778e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:18.822026 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edaaa8c-dae1-4741-aa37-87553151778e admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:18.822303 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edaaa8c-dae1-4741-aa37-87553151778e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:18.822568 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edaaa8c-dae1-4741-aa37-87553151778e admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:18.823226 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edaaa8c-dae1-4741-aa37-87553151778e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:18.825513 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edaaa8c-dae1-4741-aa37-87553151778e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:18.854366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1357/2715] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:28:18 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:28:18.858647 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a680e-e9ae-432a-9feb-5761032d06dd admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:18.869671 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a680e-e9ae-432a-9feb-5761032d06dd admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:18.928014 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce0a8a-3d63-48e2-8dbe-4bfc0209e8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:18.928509 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce0a8a-3d63-48e2-8dbe-4bfc0209e8ab admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:18.928785 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce0a8a-3d63-48e2-8dbe-4bfc0209e8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:18.929055 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce0a8a-3d63-48e2-8dbe-4bfc0209e8ab admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:18.929695 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce0a8a-3d63-48e2-8dbe-4bfc0209e8ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:18.931654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce0a8a-3d63-48e2-8dbe-4bfc0209e8ab admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:18.952914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1359/2716] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:28:18 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:18.958097 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edaaa8c-dae1-4741-aa37-87553151778e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:18.968571 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edaaa8c-dae1-4741-aa37-87553151778e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:18.973966 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da9c8a-8145-4470-a820-66f452890ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:18.974562 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da9c8a-8145-4470-a820-66f452890ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:18.974874 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da9c8a-8145-4470-a820-66f452890ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:18.975152 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da9c8a-8145-4470-a820-66f452890ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:18.975788 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da9c8a-8145-4470-a820-66f452890ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:18.977384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da9c8a-8145-4470-a820-66f452890ef7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:18.987470 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1358/2717] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:18.991492 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce0a8a-3d63-48e2-8dbe-4bfc0209e8ab admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:19.002283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce0a8a-3d63-48e2-8dbe-4bfc0209e8ab admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:19.007095 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9255e-e56c-4528-8d06-c4f3ecd6585b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:19.007684 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9255e-e56c-4528-8d06-c4f3ecd6585b admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:19.007959 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9255e-e56c-4528-8d06-c4f3ecd6585b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:19.008228 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9255e-e56c-4528-8d06-c4f3ecd6585b admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:19.008890 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9255e-e56c-4528-8d06-c4f3ecd6585b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:19.010519 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9255e-e56c-4528-8d06-c4f3ecd6585b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:19.021811 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1360/2718] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:19.025898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da9c8a-8145-4470-a820-66f452890ef7 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:19.036572 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da9c8a-8145-4470-a820-66f452890ef7 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:19.041339 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fd06d-3b2c-4a63-86bc-6f34eaba22dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:19.042031 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fd06d-3b2c-4a63-86bc-6f34eaba22dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:19.042349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fd06d-3b2c-4a63-86bc-6f34eaba22dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:19.042626 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fd06d-3b2c-4a63-86bc-6f34eaba22dc admin admin] PATH_INFO: `/v3/projects/c2752a21585341c8abf74ad9c45cf546/users/8e1598e960c1443eae7c0c48ff90673d/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:19.057467 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17fd06d-3b2c-4a63-86bc-6f34eaba22dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2752a21585341c8abf74ad9c45cf546, user_id=8e1598e960c1443eae7c0c48ff90673d, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:19.060163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17fd06d-3b2c-4a63-86bc-6f34eaba22dc admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:19.076749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1359/2719] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:28:19 2022] PUT /identity/v3/projects/c2752a21585341c8abf74ad9c45cf546/users/8e1598e960c1443eae7c0c48ff90673d/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:19.088193 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ac2b2-0bef-4359-ad2a-a604c58c21aa None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:19.088483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ac2b2-0bef-4359-ad2a-a604c58c21aa None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:19.088776 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ac2b2-0bef-4359-ad2a-a604c58c21aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:19.097151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:19.120641 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b37ac2b2-0bef-4359-ad2a-a604c58c21aa None None] MFA Rules not processed for user `8e1598e960c1443eae7c0c48ff90673d`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:19.121875 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b37ac2b2-0bef-4359-ad2a-a604c58c21aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:19.200501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1361/2720] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:19.212168 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fd06d-3b2c-4a63-86bc-6f34eaba22dc admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:19.213913 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c17fd06d-3b2c-4a63-86bc-6f34eaba22dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:19.273136 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fd06d-3b2c-4a63-86bc-6f34eaba22dc admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:19.277145 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207658c-5c49-48a0-8baf-b297696dd394 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:19.277746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207658c-5c49-48a0-8baf-b297696dd394 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:19.278030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207658c-5c49-48a0-8baf-b297696dd394 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:19.278300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207658c-5c49-48a0-8baf-b297696dd394 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:19.286674 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207658c-5c49-48a0-8baf-b297696dd394 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:19.288111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207658c-5c49-48a0-8baf-b297696dd394 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:19.297528 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1360/2721] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:28.398753 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ac2b2-0bef-4359-ad2a-a604c58c21aa None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:28.400148 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b37ac2b2-0bef-4359-ad2a-a604c58c21aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:28.475564 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ac2b2-0bef-4359-ad2a-a604c58c21aa None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:28.481448 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc548f-4ae9-470e-ac1d-83950858bee1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034d75ab747e4c0cab040e47c6a6236e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:28.482053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc548f-4ae9-470e-ac1d-83950858bee1 service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:28.482329 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc548f-4ae9-470e-ac1d-83950858bee1 service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:28.482597 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc548f-4ae9-470e-ac1d-83950858bee1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:28.484379 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-37cc548f-4ae9-470e-ac1d-83950858bee1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:28.497068 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc548f-4ae9-470e-ac1d-83950858bee1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:28.498621 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc548f-4ae9-470e-ac1d-83950858bee1 service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:28.510595 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1362/2722] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:28.774960 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207658c-5c49-48a0-8baf-b297696dd394 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:28.839384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207658c-5c49-48a0-8baf-b297696dd394 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:28.843914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbef2ea-7ad1-4da6-b9ab-ab998c750482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:28.844531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbef2ea-7ad1-4da6-b9ab-ab998c750482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:28.844806 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbef2ea-7ad1-4da6-b9ab-ab998c750482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:28.845075 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbef2ea-7ad1-4da6-b9ab-ab998c750482 admin admin] PATH_INFO: `/v3/users/8e1598e960c1443eae7c0c48ff90673d` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:28.846170 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbef2ea-7ad1-4da6-b9ab-ab998c750482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e1598e960c1443eae7c0c48ff90673d)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:28.848155 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbef2ea-7ad1-4da6-b9ab-ab998c750482 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:28.849667 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:28.895935 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-afbef2ea-7ad1-4da6-b9ab-ab998c750482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e1598e960c1443eae7c0c48ff90673d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:28.902633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-afbef2ea-7ad1-4da6-b9ab-ab998c750482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e1598e960c1443eae7c0c48ff90673d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:28.914364 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-afbef2ea-7ad1-4da6-b9ab-ab998c750482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e1598e960c1443eae7c0c48ff90673d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:28.915076 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-afbef2ea-7ad1-4da6-b9ab-ab998c750482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e1598e960c1443eae7c0c48ff90673d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:28.915682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-afbef2ea-7ad1-4da6-b9ab-ab998c750482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e1598e960c1443eae7c0c48ff90673d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:28.929042 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1361/2723] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:28:28 2022] DELETE /identity/v3/users/8e1598e960c1443eae7c0c48ff90673d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:28.934016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1bb63-3e78-47a5-ad80-a756d227e9c3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:28.934306 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1bb63-3e78-47a5-ad80-a756d227e9c3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:28.934582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1bb63-3e78-47a5-ad80-a756d227e9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:28.954685 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-5fa1bb63-3e78-47a5-ad80-a756d227e9c3 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:28.955983 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5fa1bb63-3e78-47a5-ad80-a756d227e9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:29.016706 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1363/2724] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:29.027232 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbef2ea-7ad1-4da6-b9ab-ab998c750482 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:29.028639 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-afbef2ea-7ad1-4da6-b9ab-ab998c750482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:29.088596 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbef2ea-7ad1-4da6-b9ab-ab998c750482 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:29.092269 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c06a5-e91e-46da-b682-d4769a63f32c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:29.092874 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c06a5-e91e-46da-b682-d4769a63f32c service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:29.093157 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c06a5-e91e-46da-b682-d4769a63f32c service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:29.093434 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c06a5-e91e-46da-b682-d4769a63f32c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:29.103140 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c06a5-e91e-46da-b682-d4769a63f32c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:29.104568 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c06a5-e91e-46da-b682-d4769a63f32c service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:29.114984 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1362/2725] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:29.390511 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1bb63-3e78-47a5-ad80-a756d227e9c3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:29.392089 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5fa1bb63-3e78-47a5-ad80-a756d227e9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:29.405131 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1bb63-3e78-47a5-ad80-a756d227e9c3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:29.411967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1308ce2-583c-4c2a-bf20-175353abc88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:29.412632 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1308ce2-583c-4c2a-bf20-175353abc88b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:29.412915 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1308ce2-583c-4c2a-bf20-175353abc88b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:29.413191 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1308ce2-583c-4c2a-bf20-175353abc88b admin admin] PATH_INFO: `/v3/projects/c2752a21585341c8abf74ad9c45cf546` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:29.414640 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1308ce2-583c-4c2a-bf20-175353abc88b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2752a21585341c8abf74ad9c45cf546)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:29.416749 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1308ce2-583c-4c2a-bf20-175353abc88b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:29.444089 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:29.473490 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c1308ce2-583c-4c2a-bf20-175353abc88b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2752a21585341c8abf74ad9c45cf546', 'request_id': 'req-c1308ce2-583c-4c2a-bf20-175353abc88b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:29.480028 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c1308ce2-583c-4c2a-bf20-175353abc88b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2752a21585341c8abf74ad9c45cf546', 'request_id': 'req-c1308ce2-583c-4c2a-bf20-175353abc88b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:29.485735 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c1308ce2-583c-4c2a-bf20-175353abc88b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2752a21585341c8abf74ad9c45cf546', 'request_id': 'req-c1308ce2-583c-4c2a-bf20-175353abc88b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:29.500158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c1308ce2-583c-4c2a-bf20-175353abc88b admin admin] The token cache is being invalidate because project c2752a21585341c8abf74ad9c45cf546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:28:29.500569 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c1308ce2-583c-4c2a-bf20-175353abc88b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:29.501284 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c1308ce2-583c-4c2a-bf20-175353abc88b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:29.502928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1364/2726] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:28:29 2022] DELETE /identity/v3/projects/c2752a21585341c8abf74ad9c45cf546 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:29.510920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b6848-ea24-44f8-8a73-455d994692b6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:29.511978 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b6848-ea24-44f8-8a73-455d994692b6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:29.512279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b6848-ea24-44f8-8a73-455d994692b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:29.533869 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-af8b6848-ea24-44f8-8a73-455d994692b6 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:29.535631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-af8b6848-ea24-44f8-8a73-455d994692b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:29.600059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1363/2727] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:29.608941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f2e22-d25d-46d1-a70f-9dc7f36af0be None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:29.609231 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f2e22-d25d-46d1-a70f-9dc7f36af0be None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:29.609507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f2e22-d25d-46d1-a70f-9dc7f36af0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:29.631817 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c66f2e22-d25d-46d1-a70f-9dc7f36af0be None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:29.633082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c66f2e22-d25d-46d1-a70f-9dc7f36af0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:29.642076 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1365/2728] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:29.646292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b6848-ea24-44f8-8a73-455d994692b6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:29.656163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b6848-ea24-44f8-8a73-455d994692b6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:29.661416 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5704b-d831-420b-822d-687c77a2e5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:29.662011 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5704b-d831-420b-822d-687c77a2e5de admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:29.662292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5704b-d831-420b-822d-687c77a2e5de admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:29.662575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5704b-d831-420b-822d-687c77a2e5de admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:29.663256 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5704b-d831-420b-822d-687c77a2e5de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:28:29.663566 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5704b-d831-420b-822d-687c77a2e5de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:29.665166 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5704b-d831-420b-822d-687c77a2e5de admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:29.677422 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1364/2729] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:28:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:28:29.681755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f2e22-d25d-46d1-a70f-9dc7f36af0be None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:29.691791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f2e22-d25d-46d1-a70f-9dc7f36af0be None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:29.696670 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececa3a8-cd98-40de-ac5e-9ba1c8a5fa10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:29.697302 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececa3a8-cd98-40de-ac5e-9ba1c8a5fa10 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:29.697588 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececa3a8-cd98-40de-ac5e-9ba1c8a5fa10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:29.697859 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececa3a8-cd98-40de-ac5e-9ba1c8a5fa10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:29.698491 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ececa3a8-cd98-40de-ac5e-9ba1c8a5fa10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:29.700442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ececa3a8-cd98-40de-ac5e-9ba1c8a5fa10 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:29.728066 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1366/2730] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:28:29 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:29.732203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5704b-d831-420b-822d-687c77a2e5de admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:29.743119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5704b-d831-420b-822d-687c77a2e5de admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:29.747793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:29.804265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835716d8-4cff-448b-b0b9-bd755f3eee96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:29.804927 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835716d8-4cff-448b-b0b9-bd755f3eee96 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:29.805086 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835716d8-4cff-448b-b0b9-bd755f3eee96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:29.805361 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835716d8-4cff-448b-b0b9-bd755f3eee96 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:29.805991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835716d8-4cff-448b-b0b9-bd755f3eee96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:29.808255 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835716d8-4cff-448b-b0b9-bd755f3eee96 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:29.831493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1365/2731] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:28:29 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:29.836955 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececa3a8-cd98-40de-ac5e-9ba1c8a5fa10 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:29.847779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececa3a8-cd98-40de-ac5e-9ba1c8a5fa10 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:29.852543 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab4194-82aa-46b1-8824-8dbdc8b451d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:29.853138 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab4194-82aa-46b1-8824-8dbdc8b451d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:29.853415 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab4194-82aa-46b1-8824-8dbdc8b451d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:29.853686 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab4194-82aa-46b1-8824-8dbdc8b451d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:29.854328 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab4194-82aa-46b1-8824-8dbdc8b451d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:29.855970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab4194-82aa-46b1-8824-8dbdc8b451d6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:29.867359 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1367/2732] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:29.871897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835716d8-4cff-448b-b0b9-bd755f3eee96 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:29.883859 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835716d8-4cff-448b-b0b9-bd755f3eee96 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:29.888923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befec0c-b7fe-44b8-9f7a-38b2adb2a640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:29.889757 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befec0c-b7fe-44b8-9f7a-38b2adb2a640 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:29.890051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befec0c-b7fe-44b8-9f7a-38b2adb2a640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:29.890329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befec0c-b7fe-44b8-9f7a-38b2adb2a640 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:29.891088 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3befec0c-b7fe-44b8-9f7a-38b2adb2a640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:29.892797 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3befec0c-b7fe-44b8-9f7a-38b2adb2a640 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:29.905517 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1366/2733] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:29.910230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab4194-82aa-46b1-8824-8dbdc8b451d6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:29.920369 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab4194-82aa-46b1-8824-8dbdc8b451d6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:29.925116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf038f-53f9-4b5b-bccd-752967bbedb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:29.926461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf038f-53f9-4b5b-bccd-752967bbedb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:29.926760 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf038f-53f9-4b5b-bccd-752967bbedb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:29.927034 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf038f-53f9-4b5b-bccd-752967bbedb6 admin admin] PATH_INFO: `/v3/projects/7af33a60ce1e4f33b86deeb093fd672c/users/b87291a65b4442d697debf623feda0e1/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:29.942573 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf038f-53f9-4b5b-bccd-752967bbedb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7af33a60ce1e4f33b86deeb093fd672c, user_id=b87291a65b4442d697debf623feda0e1, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:29.945261 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf038f-53f9-4b5b-bccd-752967bbedb6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:29.958800 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1368/2734] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:28:29 2022] PUT /identity/v3/projects/7af33a60ce1e4f33b86deeb093fd672c/users/b87291a65b4442d697debf623feda0e1/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:29.969192 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df7896-dfed-4ccb-a35a-0cce0c4c0ca8 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:29.969479 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df7896-dfed-4ccb-a35a-0cce0c4c0ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:29.969753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df7896-dfed-4ccb-a35a-0cce0c4c0ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:29.999023 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-54df7896-dfed-4ccb-a35a-0cce0c4c0ca8 None None] MFA Rules not processed for user `b87291a65b4442d697debf623feda0e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:30.000227 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-54df7896-dfed-4ccb-a35a-0cce0c4c0ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:30.067791 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1367/2735] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:30.072524 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf038f-53f9-4b5b-bccd-752967bbedb6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:30.136339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf038f-53f9-4b5b-bccd-752967bbedb6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:30.141339 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717124d1-dc7b-4ed6-aacd-3c274532e07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:30.141947 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717124d1-dc7b-4ed6-aacd-3c274532e07d admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:30.142229 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717124d1-dc7b-4ed6-aacd-3c274532e07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:30.142501 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717124d1-dc7b-4ed6-aacd-3c274532e07d admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:30.143164 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717124d1-dc7b-4ed6-aacd-3c274532e07d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:30.146187 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717124d1-dc7b-4ed6-aacd-3c274532e07d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:30.175182 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1369/2736] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:28:30 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:30.179448 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df7896-dfed-4ccb-a35a-0cce0c4c0ca8 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:30.190550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df7896-dfed-4ccb-a35a-0cce0c4c0ca8 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:30.255503 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d1e22-cc11-4bf1-9034-060cbe9f31ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:30.256124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d1e22-cc11-4bf1-9034-060cbe9f31ed admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:30.256402 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d1e22-cc11-4bf1-9034-060cbe9f31ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:30.256672 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d1e22-cc11-4bf1-9034-060cbe9f31ed admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:30.257353 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d1e22-cc11-4bf1-9034-060cbe9f31ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:30.259374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d1e22-cc11-4bf1-9034-060cbe9f31ed admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:30.284866 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1368/2737] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:28:30 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:30.286847 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717124d1-dc7b-4ed6-aacd-3c274532e07d admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:30.313272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717124d1-dc7b-4ed6-aacd-3c274532e07d admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:30.321471 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ffbb1-ea4d-4e5e-9180-0b66532ec4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:30.321947 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ffbb1-ea4d-4e5e-9180-0b66532ec4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:30.322226 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ffbb1-ea4d-4e5e-9180-0b66532ec4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:30.323055 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ffbb1-ea4d-4e5e-9180-0b66532ec4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:30.323194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ffbb1-ea4d-4e5e-9180-0b66532ec4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:30.324815 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ffbb1-ea4d-4e5e-9180-0b66532ec4d3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:30.336930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1370/2738] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:30.340969 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d1e22-cc11-4bf1-9034-060cbe9f31ed admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:30.351877 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d1e22-cc11-4bf1-9034-060cbe9f31ed admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:30.356603 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a320c9-422f-4af2-8bdc-1d4dddc36199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:30.357213 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a320c9-422f-4af2-8bdc-1d4dddc36199 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:30.357493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a320c9-422f-4af2-8bdc-1d4dddc36199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:30.357763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a320c9-422f-4af2-8bdc-1d4dddc36199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:30.358411 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a320c9-422f-4af2-8bdc-1d4dddc36199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:30.360079 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a320c9-422f-4af2-8bdc-1d4dddc36199 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:30.361530 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:30.372630 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1369/2739] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:30.376563 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ffbb1-ea4d-4e5e-9180-0b66532ec4d3 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:30.387824 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ffbb1-ea4d-4e5e-9180-0b66532ec4d3 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:30.393150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6149e6-f5a3-46b9-9d65-eb46f882995f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:30.393805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6149e6-f5a3-46b9-9d65-eb46f882995f admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:30.394599 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6149e6-f5a3-46b9-9d65-eb46f882995f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:30.395030 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6149e6-f5a3-46b9-9d65-eb46f882995f admin admin] PATH_INFO: `/v3/projects/35b3d1b16555497e9b2b2fef24c3680d/users/3e059035278946908ef54ee119e6d20b/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:30.410816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6149e6-f5a3-46b9-9d65-eb46f882995f admin admin] RBAC: Authorizing `identity:create_grant(project_id=35b3d1b16555497e9b2b2fef24c3680d, user_id=3e059035278946908ef54ee119e6d20b, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:30.413586 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6149e6-f5a3-46b9-9d65-eb46f882995f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:30.415936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:30.430419 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1371/2740] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:28:30 2022] PUT /identity/v3/projects/35b3d1b16555497e9b2b2fef24c3680d/users/3e059035278946908ef54ee119e6d20b/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:30.440607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94914d-241a-4171-902a-52dc48487865 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:30.440971 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94914d-241a-4171-902a-52dc48487865 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:30.440971 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94914d-241a-4171-902a-52dc48487865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:30.479113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-fe94914d-241a-4171-902a-52dc48487865 None None] MFA Rules not processed for user `3e059035278946908ef54ee119e6d20b`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:30.480340 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fe94914d-241a-4171-902a-52dc48487865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:30.552126 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1370/2741] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:30.556566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6149e6-f5a3-46b9-9d65-eb46f882995f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:30.624737 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6149e6-f5a3-46b9-9d65-eb46f882995f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:30.630281 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a7d1f-94f2-4a63-84c3-d826eb1493c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:30.630975 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a7d1f-94f2-4a63-84c3-d826eb1493c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:30.631260 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a7d1f-94f2-4a63-84c3-d826eb1493c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:30.631531 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a7d1f-94f2-4a63-84c3-d826eb1493c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:30.632178 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a7d1f-94f2-4a63-84c3-d826eb1493c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:30.634171 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a7d1f-94f2-4a63-84c3-d826eb1493c1 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:30.673598 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1372/2742] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:28:30 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:30.675128 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94914d-241a-4171-902a-52dc48487865 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:30.688170 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94914d-241a-4171-902a-52dc48487865 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:30.755097 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a4241-9cb5-423d-8fe8-3982b0ecca65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:30.755716 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a4241-9cb5-423d-8fe8-3982b0ecca65 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:30.756000 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a4241-9cb5-423d-8fe8-3982b0ecca65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:30.756276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a4241-9cb5-423d-8fe8-3982b0ecca65 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:30.756925 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a4241-9cb5-423d-8fe8-3982b0ecca65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:30.758905 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a4241-9cb5-423d-8fe8-3982b0ecca65 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:30.780671 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1371/2743] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:28:30 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:30.784638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a7d1f-94f2-4a63-84c3-d826eb1493c1 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:30.796577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a7d1f-94f2-4a63-84c3-d826eb1493c1 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:30.801860 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6844052-7922-49a0-8852-41da7580a256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:30.802465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6844052-7922-49a0-8852-41da7580a256 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:30.802788 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6844052-7922-49a0-8852-41da7580a256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:30.803065 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6844052-7922-49a0-8852-41da7580a256 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:30.803718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6844052-7922-49a0-8852-41da7580a256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:30.805300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6844052-7922-49a0-8852-41da7580a256 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:30.816293 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1373/2744] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:28:30.821187 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a4241-9cb5-423d-8fe8-3982b0ecca65 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:30.833417 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a4241-9cb5-423d-8fe8-3982b0ecca65 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:30.838413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bd967-d1dc-4898-bc65-5b1a7a0f7f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:30.839137 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bd967-d1dc-4898-bc65-5b1a7a0f7f5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:30.839417 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bd967-d1dc-4898-bc65-5b1a7a0f7f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:30.839691 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bd967-d1dc-4898-bc65-5b1a7a0f7f5c admin admin] PATH_INFO: `/v3/projects/a69dbde96a434675b07ea5d5c0754c79/users/9ccde5530d3047248cc881080200902c/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:30.856537 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0bd967-d1dc-4898-bc65-5b1a7a0f7f5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a69dbde96a434675b07ea5d5c0754c79, user_id=9ccde5530d3047248cc881080200902c, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:30.859283 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0bd967-d1dc-4898-bc65-5b1a7a0f7f5c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:30.877669 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1372/2745] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:28:30 2022] PUT /identity/v3/projects/a69dbde96a434675b07ea5d5c0754c79/users/9ccde5530d3047248cc881080200902c/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:28:30.881783 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6844052-7922-49a0-8852-41da7580a256 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:30.892384 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6844052-7922-49a0-8852-41da7580a256 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:30.952020 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69d625-783d-4126-ae35-13f184b7d620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:30.952625 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69d625-783d-4126-ae35-13f184b7d620 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:30.952933 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69d625-783d-4126-ae35-13f184b7d620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:30.953207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69d625-783d-4126-ae35-13f184b7d620 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:30.954165 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae69d625-783d-4126-ae35-13f184b7d620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:30.955927 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae69d625-783d-4126-ae35-13f184b7d620 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:30.966504 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1374/2746] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:30.970781 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bd967-d1dc-4898-bc65-5b1a7a0f7f5c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:30.981877 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bd967-d1dc-4898-bc65-5b1a7a0f7f5c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:30.986553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7dcb02-4f23-4b77-8be9-5b04261b7c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:30.987237 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7dcb02-4f23-4b77-8be9-5b04261b7c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:30.987517 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7dcb02-4f23-4b77-8be9-5b04261b7c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:30.987823 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7dcb02-4f23-4b77-8be9-5b04261b7c5e admin admin] PATH_INFO: `/v3/projects/a69dbde96a434675b07ea5d5c0754c79/users/9ccde5530d3047248cc881080200902c/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:30.989749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7dcb02-4f23-4b77-8be9-5b04261b7c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a69dbde96a434675b07ea5d5c0754c79, user_id=9ccde5530d3047248cc881080200902c, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:30.992451 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7dcb02-4f23-4b77-8be9-5b04261b7c5e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:31.003544 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1373/2747] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:28:30 2022] PUT /identity/v3/projects/a69dbde96a434675b07ea5d5c0754c79/users/9ccde5530d3047248cc881080200902c/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:31.080715 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5205e4-7cd0-40f7-a0e3-daf3b3ac7227 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:31.080999 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5205e4-7cd0-40f7-a0e3-daf3b3ac7227 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:31.081251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5205e4-7cd0-40f7-a0e3-daf3b3ac7227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:31.109984 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4f5205e4-7cd0-40f7-a0e3-daf3b3ac7227 None None] MFA Rules not processed for user `9ccde5530d3047248cc881080200902c`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:31.111131 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4f5205e4-7cd0-40f7-a0e3-daf3b3ac7227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:31.163366 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:31.183939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1375/2748] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:31 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:28:31.196454 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7dcb02-4f23-4b77-8be9-5b04261b7c5e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:31.198160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fa7dcb02-4f23-4b77-8be9-5b04261b7c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:31.241642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:31.259695 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7dcb02-4f23-4b77-8be9-5b04261b7c5e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:31.263393 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac422eea-66b0-4ca3-a5bf-51efd2790424 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:31.264001 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac422eea-66b0-4ca3-a5bf-51efd2790424 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:31.264320 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac422eea-66b0-4ca3-a5bf-51efd2790424 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:31.264575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac422eea-66b0-4ca3-a5bf-51efd2790424 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:31.323388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac422eea-66b0-4ca3-a5bf-51efd2790424 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:31.324827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac422eea-66b0-4ca3-a5bf-51efd2790424 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:31.334653 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1374/2749] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:32.850719 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5205e4-7cd0-40f7-a0e3-daf3b3ac7227 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:32.852212 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4f5205e4-7cd0-40f7-a0e3-daf3b3ac7227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:32.864921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5205e4-7cd0-40f7-a0e3-daf3b3ac7227 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:32.868524 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457e439-aced-43eb-900b-6ab0ba717db0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:32.869120 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457e439-aced-43eb-900b-6ab0ba717db0 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:32.869396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457e439-aced-43eb-900b-6ab0ba717db0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:32.869668 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457e439-aced-43eb-900b-6ab0ba717db0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:32.933402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457e439-aced-43eb-900b-6ab0ba717db0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:32.934866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457e439-aced-43eb-900b-6ab0ba717db0 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:32.943718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1376/2750] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:33.486166 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac422eea-66b0-4ca3-a5bf-51efd2790424 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:33.494930 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac422eea-66b0-4ca3-a5bf-51efd2790424 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:33.498734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53713c0f-be1f-4a74-9ad3-a135e6aec551 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:33.499366 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53713c0f-be1f-4a74-9ad3-a135e6aec551 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:33.499676 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53713c0f-be1f-4a74-9ad3-a135e6aec551 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:33.499945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53713c0f-be1f-4a74-9ad3-a135e6aec551 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:33.509806 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53713c0f-be1f-4a74-9ad3-a135e6aec551 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:33.511281 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53713c0f-be1f-4a74-9ad3-a135e6aec551 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:33.521319 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1375/2751] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:37.452882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457e439-aced-43eb-900b-6ab0ba717db0 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:37.518907 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457e439-aced-43eb-900b-6ab0ba717db0 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:37.523433 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcf6e0-9c4a-404e-a6c4-a1fbad7cbed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:37.524047 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcf6e0-9c4a-404e-a6c4-a1fbad7cbed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:37.524320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcf6e0-9c4a-404e-a6c4-a1fbad7cbed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:37.524586 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcf6e0-9c4a-404e-a6c4-a1fbad7cbed1 admin admin] PATH_INFO: `/v3/users/b87291a65b4442d697debf623feda0e1` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:37.525683 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dcf6e0-9c4a-404e-a6c4-a1fbad7cbed1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b87291a65b4442d697debf623feda0e1)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:37.527642 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dcf6e0-9c4a-404e-a6c4-a1fbad7cbed1 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:37.574422 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c3dcf6e0-9c4a-404e-a6c4-a1fbad7cbed1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b87291a65b4442d697debf623feda0e1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:37.581797 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c3dcf6e0-9c4a-404e-a6c4-a1fbad7cbed1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b87291a65b4442d697debf623feda0e1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:37.595018 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c3dcf6e0-9c4a-404e-a6c4-a1fbad7cbed1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b87291a65b4442d697debf623feda0e1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:37.595707 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c3dcf6e0-9c4a-404e-a6c4-a1fbad7cbed1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b87291a65b4442d697debf623feda0e1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:37.605668 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c3dcf6e0-9c4a-404e-a6c4-a1fbad7cbed1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b87291a65b4442d697debf623feda0e1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:37.611841 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1377/2752] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:28:37 2022] DELETE /identity/v3/users/b87291a65b4442d697debf623feda0e1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:28:37.616360 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc79105-6828-4555-af47-3b2ab210fdde None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:37.616843 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc79105-6828-4555-af47-3b2ab210fdde None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:37.616946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc79105-6828-4555-af47-3b2ab210fdde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:37.636962 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-3dc79105-6828-4555-af47-3b2ab210fdde None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:37.636962 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3dc79105-6828-4555-af47-3b2ab210fdde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:37.698682 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1376/2753] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:37.709387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcf6e0-9c4a-404e-a6c4-a1fbad7cbed1 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:37.710875 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c3dcf6e0-9c4a-404e-a6c4-a1fbad7cbed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:37.720942 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:37.770843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcf6e0-9c4a-404e-a6c4-a1fbad7cbed1 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:37.774602 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809fd46-7c31-49c7-a129-3d135fc933d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:37.775233 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809fd46-7c31-49c7-a129-3d135fc933d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:37.775515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809fd46-7c31-49c7-a129-3d135fc933d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:37.775789 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809fd46-7c31-49c7-a129-3d135fc933d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:37.784875 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809fd46-7c31-49c7-a129-3d135fc933d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:37.787522 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809fd46-7c31-49c7-a129-3d135fc933d8 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:37.797405 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1378/2754] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:38.069349 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc79105-6828-4555-af47-3b2ab210fdde None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:38.070839 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3dc79105-6828-4555-af47-3b2ab210fdde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:38.086002 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc79105-6828-4555-af47-3b2ab210fdde None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:38.090618 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2486471-3d6e-44ec-a234-6acff200e8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:38.091815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2486471-3d6e-44ec-a234-6acff200e8c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:38.092090 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2486471-3d6e-44ec-a234-6acff200e8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:38.092360 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2486471-3d6e-44ec-a234-6acff200e8c1 admin admin] PATH_INFO: `/v3/projects/7af33a60ce1e4f33b86deeb093fd672c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:38.093683 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2486471-3d6e-44ec-a234-6acff200e8c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7af33a60ce1e4f33b86deeb093fd672c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:38.095769 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2486471-3d6e-44ec-a234-6acff200e8c1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:38.144576 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f2486471-3d6e-44ec-a234-6acff200e8c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7af33a60ce1e4f33b86deeb093fd672c', 'request_id': 'req-f2486471-3d6e-44ec-a234-6acff200e8c1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.153125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f2486471-3d6e-44ec-a234-6acff200e8c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7af33a60ce1e4f33b86deeb093fd672c', 'request_id': 'req-f2486471-3d6e-44ec-a234-6acff200e8c1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.159178 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f2486471-3d6e-44ec-a234-6acff200e8c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7af33a60ce1e4f33b86deeb093fd672c', 'request_id': 'req-f2486471-3d6e-44ec-a234-6acff200e8c1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.170923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f2486471-3d6e-44ec-a234-6acff200e8c1 admin admin] The token cache is being invalidate because project 7af33a60ce1e4f33b86deeb093fd672c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:28:38.171214 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f2486471-3d6e-44ec-a234-6acff200e8c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.171849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f2486471-3d6e-44ec-a234-6acff200e8c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.173504 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1377/2755] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:28:38 2022] DELETE /identity/v3/projects/7af33a60ce1e4f33b86deeb093fd672c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:38.177550 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809fd46-7c31-49c7-a129-3d135fc933d8 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:38.187350 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809fd46-7c31-49c7-a129-3d135fc933d8 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:38.245306 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2bd3a-3b68-44a6-860b-5aaf298af1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:38.245945 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2bd3a-3b68-44a6-860b-5aaf298af1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:38.246227 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2bd3a-3b68-44a6-860b-5aaf298af1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:38.246503 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2bd3a-3b68-44a6-860b-5aaf298af1e4 admin admin] PATH_INFO: `/v3/users/3e059035278946908ef54ee119e6d20b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:38.247735 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2bd3a-3b68-44a6-860b-5aaf298af1e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e059035278946908ef54ee119e6d20b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:38.249847 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2bd3a-3b68-44a6-860b-5aaf298af1e4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:38.299584 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e1d2bd3a-3b68-44a6-860b-5aaf298af1e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e059035278946908ef54ee119e6d20b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.306831 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e1d2bd3a-3b68-44a6-860b-5aaf298af1e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e059035278946908ef54ee119e6d20b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.319564 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e1d2bd3a-3b68-44a6-860b-5aaf298af1e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e059035278946908ef54ee119e6d20b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.320286 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e1d2bd3a-3b68-44a6-860b-5aaf298af1e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e059035278946908ef54ee119e6d20b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.327732 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e1d2bd3a-3b68-44a6-860b-5aaf298af1e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e059035278946908ef54ee119e6d20b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.334746 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1379/2756] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:28:38 2022] DELETE /identity/v3/users/3e059035278946908ef54ee119e6d20b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:38.613534 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2486471-3d6e-44ec-a234-6acff200e8c1 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:38.615183 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f2486471-3d6e-44ec-a234-6acff200e8c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:38.623418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:38.630359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2486471-3d6e-44ec-a234-6acff200e8c1 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:38.691565 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3265c-a421-4ad7-912b-39c9b698a785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:38.692301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3265c-a421-4ad7-912b-39c9b698a785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:38.692742 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3265c-a421-4ad7-912b-39c9b698a785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:38.692959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3265c-a421-4ad7-912b-39c9b698a785 admin admin] PATH_INFO: `/v3/projects/35b3d1b16555497e9b2b2fef24c3680d` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:38.694112 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb3265c-a421-4ad7-912b-39c9b698a785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35b3d1b16555497e9b2b2fef24c3680d)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:38.696077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb3265c-a421-4ad7-912b-39c9b698a785 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:38.751678 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cbb3265c-a421-4ad7-912b-39c9b698a785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35b3d1b16555497e9b2b2fef24c3680d', 'request_id': 'req-cbb3265c-a421-4ad7-912b-39c9b698a785'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.760418 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cbb3265c-a421-4ad7-912b-39c9b698a785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35b3d1b16555497e9b2b2fef24c3680d', 'request_id': 'req-cbb3265c-a421-4ad7-912b-39c9b698a785'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.766123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cbb3265c-a421-4ad7-912b-39c9b698a785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35b3d1b16555497e9b2b2fef24c3680d', 'request_id': 'req-cbb3265c-a421-4ad7-912b-39c9b698a785'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.778455 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cbb3265c-a421-4ad7-912b-39c9b698a785 admin admin] The token cache is being invalidate because project 35b3d1b16555497e9b2b2fef24c3680d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:28:38.778778 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cbb3265c-a421-4ad7-912b-39c9b698a785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.779432 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-cbb3265c-a421-4ad7-912b-39c9b698a785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.780994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1378/2757] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:28:38 2022] DELETE /identity/v3/projects/35b3d1b16555497e9b2b2fef24c3680d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:28:38.785170 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2bd3a-3b68-44a6-860b-5aaf298af1e4 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:38.786779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e1d2bd3a-3b68-44a6-860b-5aaf298af1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:38.800015 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2bd3a-3b68-44a6-860b-5aaf298af1e4 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:38.829247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:38.864013 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8636d9-9679-4354-8469-444b1061b8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:38.864348 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8636d9-9679-4354-8469-444b1061b8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:38.864633 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8636d9-9679-4354-8469-444b1061b8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:38.864908 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8636d9-9679-4354-8469-444b1061b8d2 admin admin] PATH_INFO: `/v3/users/9ccde5530d3047248cc881080200902c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:38.866070 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8636d9-9679-4354-8469-444b1061b8d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ccde5530d3047248cc881080200902c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:38.868043 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8636d9-9679-4354-8469-444b1061b8d2 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:38.917987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ef8636d9-9679-4354-8469-444b1061b8d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ccde5530d3047248cc881080200902c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.925374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ef8636d9-9679-4354-8469-444b1061b8d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ccde5530d3047248cc881080200902c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.937830 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ef8636d9-9679-4354-8469-444b1061b8d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ccde5530d3047248cc881080200902c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.938566 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ef8636d9-9679-4354-8469-444b1061b8d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ccde5530d3047248cc881080200902c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.948333 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ef8636d9-9679-4354-8469-444b1061b8d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ccde5530d3047248cc881080200902c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:38.954600 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1380/2758] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:28:38 2022] DELETE /identity/v3/users/9ccde5530d3047248cc881080200902c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:28:39.231616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3265c-a421-4ad7-912b-39c9b698a785 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:39.233045 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cbb3265c-a421-4ad7-912b-39c9b698a785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:39.248647 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3265c-a421-4ad7-912b-39c9b698a785 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:39.310463 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165cdf79-b149-462c-afa6-fa4336b114a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:39.311131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cdf79-b149-462c-afa6-fa4336b114a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:39.311405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cdf79-b149-462c-afa6-fa4336b114a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:39.311675 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cdf79-b149-462c-afa6-fa4336b114a7 admin admin] PATH_INFO: `/v3/projects/a69dbde96a434675b07ea5d5c0754c79` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:39.313030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165cdf79-b149-462c-afa6-fa4336b114a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a69dbde96a434675b07ea5d5c0754c79)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:39.315181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165cdf79-b149-462c-afa6-fa4336b114a7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:39.360364 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:39.375708 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-165cdf79-b149-462c-afa6-fa4336b114a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a69dbde96a434675b07ea5d5c0754c79', 'request_id': 'req-165cdf79-b149-462c-afa6-fa4336b114a7'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:39.383209 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-165cdf79-b149-462c-afa6-fa4336b114a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a69dbde96a434675b07ea5d5c0754c79', 'request_id': 'req-165cdf79-b149-462c-afa6-fa4336b114a7'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:39.388854 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-165cdf79-b149-462c-afa6-fa4336b114a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a69dbde96a434675b07ea5d5c0754c79', 'request_id': 'req-165cdf79-b149-462c-afa6-fa4336b114a7'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:39.400977 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-165cdf79-b149-462c-afa6-fa4336b114a7 admin admin] The token cache is being invalidate because project a69dbde96a434675b07ea5d5c0754c79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:28:39.401263 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-165cdf79-b149-462c-afa6-fa4336b114a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:39.401901 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-165cdf79-b149-462c-afa6-fa4336b114a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:39.403405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1379/2759] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:28:39 2022] DELETE /identity/v3/projects/a69dbde96a434675b07ea5d5c0754c79 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:39.409167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73858d98-b556-4a9a-8f1c-e5e4814da278 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:39.409532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73858d98-b556-4a9a-8f1c-e5e4814da278 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:39.409878 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73858d98-b556-4a9a-8f1c-e5e4814da278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:39.431298 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-73858d98-b556-4a9a-8f1c-e5e4814da278 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:39.432514 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-73858d98-b556-4a9a-8f1c-e5e4814da278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:39.494374 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1381/2760] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:39.502650 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d1a41-6899-4029-8821-decd8059adcc None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:39.503115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d1a41-6899-4029-8821-decd8059adcc None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:39.503403 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d1a41-6899-4029-8821-decd8059adcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:39.525292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-785d1a41-6899-4029-8821-decd8059adcc None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:39.526508 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-785d1a41-6899-4029-8821-decd8059adcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:39.534466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1380/2761] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:39.539120 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73858d98-b556-4a9a-8f1c-e5e4814da278 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:39.549383 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73858d98-b556-4a9a-8f1c-e5e4814da278 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:39.554589 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacd4de-12a8-430c-a1cb-e81099708bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:39.555233 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacd4de-12a8-430c-a1cb-e81099708bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:39.555520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacd4de-12a8-430c-a1cb-e81099708bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:39.555796 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacd4de-12a8-430c-a1cb-e81099708bf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:39.556446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacd4de-12a8-430c-a1cb-e81099708bf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:28:39.556758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacd4de-12a8-430c-a1cb-e81099708bf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:39.558348 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacd4de-12a8-430c-a1cb-e81099708bf7 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:39.570976 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1382/2762] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:28:39.575312 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d1a41-6899-4029-8821-decd8059adcc None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:39.585997 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d1a41-6899-4029-8821-decd8059adcc None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:39.591062 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c3a7c-0cd3-416b-a3db-2eaa1cd80fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:39.591661 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c3a7c-0cd3-416b-a3db-2eaa1cd80fcd admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:39.591941 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c3a7c-0cd3-416b-a3db-2eaa1cd80fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:39.592222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c3a7c-0cd3-416b-a3db-2eaa1cd80fcd admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:39.592902 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c3a7c-0cd3-416b-a3db-2eaa1cd80fcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:39.594879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c3a7c-0cd3-416b-a3db-2eaa1cd80fcd admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:39.623273 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1381/2763] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:28:39 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:39.627455 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacd4de-12a8-430c-a1cb-e81099708bf7 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:39.638213 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacd4de-12a8-430c-a1cb-e81099708bf7 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:39.672114 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:39.699567 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b4103-93f8-4c1f-97f2-01f844bc0949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:39.700175 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b4103-93f8-4c1f-97f2-01f844bc0949 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:39.700454 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b4103-93f8-4c1f-97f2-01f844bc0949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:39.700727 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b4103-93f8-4c1f-97f2-01f844bc0949 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:39.701362 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b4103-93f8-4c1f-97f2-01f844bc0949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:39.703323 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b4103-93f8-4c1f-97f2-01f844bc0949 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:39.726648 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1383/2764] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:28:39 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:28:39.731115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c3a7c-0cd3-416b-a3db-2eaa1cd80fcd admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:39.741862 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c3a7c-0cd3-416b-a3db-2eaa1cd80fcd admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:39.746920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a199e2b-8cd3-44b8-a8e4-64a4d01dc3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:39.747502 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a199e2b-8cd3-44b8-a8e4-64a4d01dc3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:39.747784 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a199e2b-8cd3-44b8-a8e4-64a4d01dc3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:39.748069 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a199e2b-8cd3-44b8-a8e4-64a4d01dc3f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:39.748812 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a199e2b-8cd3-44b8-a8e4-64a4d01dc3f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:39.750671 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a199e2b-8cd3-44b8-a8e4-64a4d01dc3f1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:39.761756 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1382/2765] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:39.765716 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b4103-93f8-4c1f-97f2-01f844bc0949 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:39.776777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b4103-93f8-4c1f-97f2-01f844bc0949 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:39.781773 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a538d21-7442-4772-bf5b-7ae147dd6276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:39.782394 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a538d21-7442-4772-bf5b-7ae147dd6276 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:39.782676 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a538d21-7442-4772-bf5b-7ae147dd6276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:39.783019 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a538d21-7442-4772-bf5b-7ae147dd6276 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:39.783664 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a538d21-7442-4772-bf5b-7ae147dd6276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:39.785330 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a538d21-7442-4772-bf5b-7ae147dd6276 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:39.795992 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1384/2766] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:28:39.800167 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a199e2b-8cd3-44b8-a8e4-64a4d01dc3f1 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:39.810485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a199e2b-8cd3-44b8-a8e4-64a4d01dc3f1 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:39.815395 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b613d08-eaca-48ed-8571-ba4f7175b071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:39.816043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b613d08-eaca-48ed-8571-ba4f7175b071 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:39.816320 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b613d08-eaca-48ed-8571-ba4f7175b071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:39.816592 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b613d08-eaca-48ed-8571-ba4f7175b071 admin admin] PATH_INFO: `/v3/projects/5ac5b5a96980409da47aee5ba455c9c0/users/8b45db0326c34b079543dcd8680bb0f2/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:39.833417 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b613d08-eaca-48ed-8571-ba4f7175b071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac5b5a96980409da47aee5ba455c9c0, user_id=8b45db0326c34b079543dcd8680bb0f2, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:39.836104 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b613d08-eaca-48ed-8571-ba4f7175b071 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:39.849649 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1383/2767] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:28:39 2022] PUT /identity/v3/projects/5ac5b5a96980409da47aee5ba455c9c0/users/8b45db0326c34b079543dcd8680bb0f2/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:39.861286 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a149ab-70e0-41a5-8a71-af2bfcf6ef3f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:39.861574 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a149ab-70e0-41a5-8a71-af2bfcf6ef3f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:39.861851 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a149ab-70e0-41a5-8a71-af2bfcf6ef3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:39.893058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-35a149ab-70e0-41a5-8a71-af2bfcf6ef3f None None] MFA Rules not processed for user `8b45db0326c34b079543dcd8680bb0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:39.894285 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-35a149ab-70e0-41a5-8a71-af2bfcf6ef3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:39.961877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1385/2768] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:28:39.973831 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b613d08-eaca-48ed-8571-ba4f7175b071 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:39.975575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1b613d08-eaca-48ed-8571-ba4f7175b071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:40.034225 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b613d08-eaca-48ed-8571-ba4f7175b071 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:40.037857 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24deba5-8643-46bd-a5fb-440cda48b6f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:40.038459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24deba5-8643-46bd-a5fb-440cda48b6f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:40.038782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24deba5-8643-46bd-a5fb-440cda48b6f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:40.039078 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24deba5-8643-46bd-a5fb-440cda48b6f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:40.047310 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24deba5-8643-46bd-a5fb-440cda48b6f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:40.048748 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24deba5-8643-46bd-a5fb-440cda48b6f6 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:40.058010 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1384/2769] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:42.629698 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a149ab-70e0-41a5-8a71-af2bfcf6ef3f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:42.692036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a149ab-70e0-41a5-8a71-af2bfcf6ef3f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:42.696648 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0247143-7f12-4f34-a4e6-6059935bf118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:42.697266 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0247143-7f12-4f34-a4e6-6059935bf118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:42.697538 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0247143-7f12-4f34-a4e6-6059935bf118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:42.697805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0247143-7f12-4f34-a4e6-6059935bf118 admin admin] PATH_INFO: `/v3/users/8b45db0326c34b079543dcd8680bb0f2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:42.698942 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0247143-7f12-4f34-a4e6-6059935bf118 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b45db0326c34b079543dcd8680bb0f2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:42.700956 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0247143-7f12-4f34-a4e6-6059935bf118 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:42.746786 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:42.748483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-f0247143-7f12-4f34-a4e6-6059935bf118 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b45db0326c34b079543dcd8680bb0f2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:42.763145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-f0247143-7f12-4f34-a4e6-6059935bf118 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b45db0326c34b079543dcd8680bb0f2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:42.775389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-f0247143-7f12-4f34-a4e6-6059935bf118 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b45db0326c34b079543dcd8680bb0f2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:42.776113 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-f0247143-7f12-4f34-a4e6-6059935bf118 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b45db0326c34b079543dcd8680bb0f2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:42.782968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-f0247143-7f12-4f34-a4e6-6059935bf118 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b45db0326c34b079543dcd8680bb0f2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:42.789449 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1386/2770] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:28:42 2022] DELETE /identity/v3/users/8b45db0326c34b079543dcd8680bb0f2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:28:42.793989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6e851-a3b6-408e-be10-e2c7a77c3bbe None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:42.794481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6e851-a3b6-408e-be10-e2c7a77c3bbe None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:42.794611 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6e851-a3b6-408e-be10-e2c7a77c3bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:42.814749 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1da6e851-a3b6-408e-be10-e2c7a77c3bbe None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:42.815959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1da6e851-a3b6-408e-be10-e2c7a77c3bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:42.875568 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1385/2771] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:28:42.886272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0247143-7f12-4f34-a4e6-6059935bf118 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:42.887823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f0247143-7f12-4f34-a4e6-6059935bf118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:42.951119 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0247143-7f12-4f34-a4e6-6059935bf118 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:42.955411 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9f251-5517-45dc-88ce-c32160dbff7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:42.956019 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9f251-5517-45dc-88ce-c32160dbff7a service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:42.956322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9f251-5517-45dc-88ce-c32160dbff7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:42.956598 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9f251-5517-45dc-88ce-c32160dbff7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:42.967233 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9f251-5517-45dc-88ce-c32160dbff7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:42.968674 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9f251-5517-45dc-88ce-c32160dbff7a service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:42.979428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1387/2772] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:28:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:28:43.269768 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6e851-a3b6-408e-be10-e2c7a77c3bbe None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:28:43.271780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1da6e851-a3b6-408e-be10-e2c7a77c3bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:43.278897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:43.284603 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6e851-a3b6-408e-be10-e2c7a77c3bbe None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:28:43.291220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8447a2-7c03-43d7-bf92-e8c2acf027db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:28:43.292054 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8447a2-7c03-43d7-bf92-e8c2acf027db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:43.292730 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8447a2-7c03-43d7-bf92-e8c2acf027db admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:43.293190 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8447a2-7c03-43d7-bf92-e8c2acf027db admin admin] PATH_INFO: `/v3/projects/5ac5b5a96980409da47aee5ba455c9c0` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:43.294644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8447a2-7c03-43d7-bf92-e8c2acf027db admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ac5b5a96980409da47aee5ba455c9c0)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:28:43.296799 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8447a2-7c03-43d7-bf92-e8c2acf027db admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:28:43.346766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7f8447a2-7c03-43d7-bf92-e8c2acf027db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ac5b5a96980409da47aee5ba455c9c0', 'request_id': 'req-7f8447a2-7c03-43d7-bf92-e8c2acf027db'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:43.354633 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7f8447a2-7c03-43d7-bf92-e8c2acf027db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ac5b5a96980409da47aee5ba455c9c0', 'request_id': 'req-7f8447a2-7c03-43d7-bf92-e8c2acf027db'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:43.360523 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7f8447a2-7c03-43d7-bf92-e8c2acf027db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ac5b5a96980409da47aee5ba455c9c0', 'request_id': 'req-7f8447a2-7c03-43d7-bf92-e8c2acf027db'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:43.371787 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7f8447a2-7c03-43d7-bf92-e8c2acf027db admin admin] The token cache is being invalidate because project 5ac5b5a96980409da47aee5ba455c9c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:28:43.372080 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7f8447a2-7c03-43d7-bf92-e8c2acf027db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:43.372746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7f8447a2-7c03-43d7-bf92-e8c2acf027db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:28:43.374224 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1386/2773] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:28:43 2022] DELETE /identity/v3/projects/5ac5b5a96980409da47aee5ba455c9c0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:28:52.974840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=39,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:53.368814 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=26,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:28:53.762704 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81c648-3c77-45e5-8301-74e670ebf36e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:28:53.762999 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81c648-3c77-45e5-8301-74e670ebf36e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:28:53.763291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81c648-3c77-45e5-8301-74e670ebf36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:28:53.783146 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-0e81c648-3c77-45e5-8301-74e670ebf36e None None] MFA Rules not processed for user `66df80c268094c48abf73f1323aa23de`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:28:53.784332 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0e81c648-3c77-45e5-8301-74e670ebf36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:28:53.840897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1388/2774] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:28:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:29:03.833563 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=25 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:30:06.836684 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8447a2-7c03-43d7-bf92-e8c2acf027db admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:06.838093 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7f8447a2-7c03-43d7-bf92-e8c2acf027db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:06.898241 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8447a2-7c03-43d7-bf92-e8c2acf027db admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:06.901770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7ce58-fcae-4404-87fc-a9a7704cb769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:06.902379 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7ce58-fcae-4404-87fc-a9a7704cb769 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:06.902674 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7ce58-fcae-4404-87fc-a9a7704cb769 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:06.902970 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7ce58-fcae-4404-87fc-a9a7704cb769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:06.904729 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ceb7ce58-fcae-4404-87fc-a9a7704cb769 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:06.980744 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7ce58-fcae-4404-87fc-a9a7704cb769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:06.982193 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7ce58-fcae-4404-87fc-a9a7704cb769 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:06.994235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1387/2775] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:30:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:30:08.178529 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81c648-3c77-45e5-8301-74e670ebf36e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:08.180075 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0e81c648-3c77-45e5-8301-74e670ebf36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:08.192188 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81c648-3c77-45e5-8301-74e670ebf36e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:08.196106 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb148e5-6536-4421-9e00-90051a5da967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:08.196711 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb148e5-6536-4421-9e00-90051a5da967 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:08.196991 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb148e5-6536-4421-9e00-90051a5da967 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:08.197266 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb148e5-6536-4421-9e00-90051a5da967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:08.199043 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9eb148e5-6536-4421-9e00-90051a5da967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:08.213315 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb148e5-6536-4421-9e00-90051a5da967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:08.214770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb148e5-6536-4421-9e00-90051a5da967 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:08.226280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1389/2776] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:30:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:30:11.663338 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de45b01-4c29-4ea0-8378-96048b3e9d06 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:11.663835 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de45b01-4c29-4ea0-8378-96048b3e9d06 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:11.664051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de45b01-4c29-4ea0-8378-96048b3e9d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:11.684949 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0de45b01-4c29-4ea0-8378-96048b3e9d06 None None] MFA Rules not processed for user `210d0f461d2c445a9688bc233d048079`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:30:11.686118 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0de45b01-4c29-4ea0-8378-96048b3e9d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:11.747359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1388/2777] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:30:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:30:12.532939 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb148e5-6536-4421-9e00-90051a5da967 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:12.534963 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9eb148e5-6536-4421-9e00-90051a5da967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:12.622570 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb148e5-6536-4421-9e00-90051a5da967 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:12.627934 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77594fe1-4ccb-4f25-9386-d0cee75462c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034d75ab747e4c0cab040e47c6a6236e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:12.628539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77594fe1-4ccb-4f25-9386-d0cee75462c0 service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:12.628820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77594fe1-4ccb-4f25-9386-d0cee75462c0 service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:12.629077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77594fe1-4ccb-4f25-9386-d0cee75462c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:12.631342 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-77594fe1-4ccb-4f25-9386-d0cee75462c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:12.643582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77594fe1-4ccb-4f25-9386-d0cee75462c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:12.646394 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77594fe1-4ccb-4f25-9386-d0cee75462c0 service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:12.656446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1390/2778] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:30:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:30:14.618609 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de45b01-4c29-4ea0-8378-96048b3e9d06 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:14.631284 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de45b01-4c29-4ea0-8378-96048b3e9d06 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:14.635030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa8b3e-fcaa-4953-bf6f-32beb4964d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:14.635657 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa8b3e-fcaa-4953-bf6f-32beb4964d94 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:14.635945 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa8b3e-fcaa-4953-bf6f-32beb4964d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:14.636383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa8b3e-fcaa-4953-bf6f-32beb4964d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:14.638206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4baa8b3e-fcaa-4953-bf6f-32beb4964d94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:14.651011 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa8b3e-fcaa-4953-bf6f-32beb4964d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:14.652838 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa8b3e-fcaa-4953-bf6f-32beb4964d94 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:14.665356 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1389/2779] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:30:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:30:15.328878 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77594fe1-4ccb-4f25-9386-d0cee75462c0 service placement] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:15.330390 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-77594fe1-4ccb-4f25-9386-d0cee75462c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:15.347135 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77594fe1-4ccb-4f25-9386-d0cee75462c0 service placement] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:15.355022 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97223bcf-aa60-43a9-92dd-f0e270fa3e60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:15.356102 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97223bcf-aa60-43a9-92dd-f0e270fa3e60 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:15.356575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97223bcf-aa60-43a9-92dd-f0e270fa3e60 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:15.357092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97223bcf-aa60-43a9-92dd-f0e270fa3e60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:15.358999 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-97223bcf-aa60-43a9-92dd-f0e270fa3e60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:15.374779 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97223bcf-aa60-43a9-92dd-f0e270fa3e60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:15.376384 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97223bcf-aa60-43a9-92dd-f0e270fa3e60 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:15.389402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1391/2780] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:30:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:30:21.393189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa8b3e-fcaa-4953-bf6f-32beb4964d94 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:21.406704 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa8b3e-fcaa-4953-bf6f-32beb4964d94 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:21.412903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9dbdf3-a959-4133-b668-84c2ceb55418 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:21.413497 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dbdf3-a959-4133-b668-84c2ceb55418 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:21.413771 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dbdf3-a959-4133-b668-84c2ceb55418 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:21.414044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dbdf3-a959-4133-b668-84c2ceb55418 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:21.415805 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1b9dbdf3-a959-4133-b668-84c2ceb55418 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:21.426315 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9dbdf3-a959-4133-b668-84c2ceb55418 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:21.430048 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9dbdf3-a959-4133-b668-84c2ceb55418 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:21.439615 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1390/2781] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:30:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:30:25.385123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=44 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:30:26.775003 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97223bcf-aa60-43a9-92dd-f0e270fa3e60 service nova] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:26.789148 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97223bcf-aa60-43a9-92dd-f0e270fa3e60 service nova] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:26.795001 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5ca2b-ea00-4bfe-9bf9-c419927514b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:26.795609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5ca2b-ea00-4bfe-9bf9-c419927514b1 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:26.795885 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5ca2b-ea00-4bfe-9bf9-c419927514b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:26.796151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5ca2b-ea00-4bfe-9bf9-c419927514b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:26.798617 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9fa5ca2b-ea00-4bfe-9bf9-c419927514b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:26.813819 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5ca2b-ea00-4bfe-9bf9-c419927514b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:26.815477 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5ca2b-ea00-4bfe-9bf9-c419927514b1 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:26.827395 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1392/2782] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:30:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:30:31.435255 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=91 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:30:36.823149 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=6 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:30:40.222583 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db840ea9-194a-41fe-90ce-6dfed0bdf0af None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:40.223163 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db840ea9-194a-41fe-90ce-6dfed0bdf0af None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:40.223322 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db840ea9-194a-41fe-90ce-6dfed0bdf0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:40.244286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-db840ea9-194a-41fe-90ce-6dfed0bdf0af None None] MFA Rules not processed for user `ba58a4914ddf418db9e8b9dce5784cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:30:40.245523 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-db840ea9-194a-41fe-90ce-6dfed0bdf0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:40.306408 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1391/2783] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 6 16:30:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:30:40.370087 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5ca2b-ea00-4bfe-9bf9-c419927514b1 service nova] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:40.371745 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9fa5ca2b-ea00-4bfe-9bf9-c419927514b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:40.453408 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5ca2b-ea00-4bfe-9bf9-c419927514b1 service nova] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:40.459132 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988dd6d6-13a3-4b33-b2de-ee37b8d55a08 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ece176d412574790a13d03dcca08f91a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:40.460962 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dd6d6-13a3-4b33-b2de-ee37b8d55a08 service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:40.461244 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dd6d6-13a3-4b33-b2de-ee37b8d55a08 service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:40.461524 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dd6d6-13a3-4b33-b2de-ee37b8d55a08 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:40.470541 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988dd6d6-13a3-4b33-b2de-ee37b8d55a08 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:40.472185 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988dd6d6-13a3-4b33-b2de-ee37b8d55a08 service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:40.481442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1393/2784] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 6 16:30:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:30:50.297329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=25 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:30:50.477908 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=32 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:30:55.489817 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db840ea9-194a-41fe-90ce-6dfed0bdf0af None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:55.503386 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db840ea9-194a-41fe-90ce-6dfed0bdf0af None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:55.508276 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c12aad-1ff4-4106-b20f-0b42fefc05f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:55.508887 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c12aad-1ff4-4106-b20f-0b42fefc05f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:55.509164 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c12aad-1ff4-4106-b20f-0b42fefc05f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:55.509434 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c12aad-1ff4-4106-b20f-0b42fefc05f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:55.511220 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c7c12aad-1ff4-4106-b20f-0b42fefc05f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:55.525015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c12aad-1ff4-4106-b20f-0b42fefc05f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:55.527913 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c12aad-1ff4-4106-b20f-0b42fefc05f7 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:55.541842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1392/2785] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:30:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:30:55.629848 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988dd6d6-13a3-4b33-b2de-ee37b8d55a08 service swift] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:55.641617 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988dd6d6-13a3-4b33-b2de-ee37b8d55a08 service swift] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:55.645343 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85b15d-30aa-4080-9222-e87031dafda5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:55.645938 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85b15d-30aa-4080-9222-e87031dafda5 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:55.646219 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85b15d-30aa-4080-9222-e87031dafda5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:55.646507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85b15d-30aa-4080-9222-e87031dafda5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:55.648895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cc85b15d-30aa-4080-9222-e87031dafda5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:55.663039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc85b15d-30aa-4080-9222-e87031dafda5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:55.664181 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc85b15d-30aa-4080-9222-e87031dafda5 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:55.675697 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1394/2786] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:30:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:30:57.472572 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3110b4-74e4-432c-ad99-bd481539910e None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:57.473089 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3110b4-74e4-432c-ad99-bd481539910e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:57.473217 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3110b4-74e4-432c-ad99-bd481539910e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:57.493429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-cf3110b4-74e4-432c-ad99-bd481539910e None None] MFA Rules not processed for user `ba58a4914ddf418db9e8b9dce5784cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:30:57.494547 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cf3110b4-74e4-432c-ad99-bd481539910e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:30:57.501800 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1393/2787] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 6 16:30:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:30:57.515820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85b15d-30aa-4080-9222-e87031dafda5 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:30:57.527556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85b15d-30aa-4080-9222-e87031dafda5 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:30:57.532594 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139ba5d-1ad1-484b-9895-aece021932e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ece176d412574790a13d03dcca08f91a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:30:57.533186 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139ba5d-1ad1-484b-9895-aece021932e6 service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:30:57.533468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139ba5d-1ad1-484b-9895-aece021932e6 service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:30:57.533745 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139ba5d-1ad1-484b-9895-aece021932e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:30:57.541845 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139ba5d-1ad1-484b-9895-aece021932e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:30:57.543539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139ba5d-1ad1-484b-9895-aece021932e6 service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:30:57.551547 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1395/2788] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 6 16:30:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 16:31:07.487620 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=9 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:31:07.548300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=12 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:31:17.072188 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f2b32-a973-4dbd-8164-3157109b27b6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:31:17.072737 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f2b32-a973-4dbd-8164-3157109b27b6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:31:17.072843 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f2b32-a973-4dbd-8164-3157109b27b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:31:17.093222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-a10f2b32-a973-4dbd-8164-3157109b27b6 None None] MFA Rules not processed for user `ba58a4914ddf418db9e8b9dce5784cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:31:17.094448 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a10f2b32-a973-4dbd-8164-3157109b27b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:31:17.102374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1394/2789] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 6 16:31:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:31:17.114895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139ba5d-1ad1-484b-9895-aece021932e6 service swift] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:31:17.126753 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139ba5d-1ad1-484b-9895-aece021932e6 service swift] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:31:17.133023 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c237c1-6774-4b65-ba63-6f9c21f96ebc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ece176d412574790a13d03dcca08f91a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:31:17.133613 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c237c1-6774-4b65-ba63-6f9c21f96ebc service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:31:17.133970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c237c1-6774-4b65-ba63-6f9c21f96ebc service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:31:17.134188 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c237c1-6774-4b65-ba63-6f9c21f96ebc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:31:17.142418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c237c1-6774-4b65-ba63-6f9c21f96ebc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:31:17.144026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c237c1-6774-4b65-ba63-6f9c21f96ebc service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:31:17.152790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1396/2790] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 6 16:31:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 16:31:27.087557 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:31:27.149487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=6 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:32:25.363120 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f2b32-a973-4dbd-8164-3157109b27b6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:25.372572 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f2b32-a973-4dbd-8164-3157109b27b6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:25.376637 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d4534-2b94-415e-afda-121ebd8aac7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:25.377236 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d4534-2b94-415e-afda-121ebd8aac7d service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:25.377514 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d4534-2b94-415e-afda-121ebd8aac7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:25.377786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d4534-2b94-415e-afda-121ebd8aac7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:25.379527 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ac3d4534-2b94-415e-afda-121ebd8aac7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:32:25.455598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d4534-2b94-415e-afda-121ebd8aac7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:25.457034 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d4534-2b94-415e-afda-121ebd8aac7d service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:25.467099 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1395/2791] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:32:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:32:28.611205 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c237c1-6774-4b65-ba63-6f9c21f96ebc service swift] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:28.613166 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-78c237c1-6774-4b65-ba63-6f9c21f96ebc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:32:28.628768 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c237c1-6774-4b65-ba63-6f9c21f96ebc service swift] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:28.634041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a146c-f27f-473d-a324-e929c135fdd3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034d75ab747e4c0cab040e47c6a6236e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:28.634684 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a146c-f27f-473d-a324-e929c135fdd3 service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:28.634968 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a146c-f27f-473d-a324-e929c135fdd3 service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:28.635341 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a146c-f27f-473d-a324-e929c135fdd3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:28.637029 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f62a146c-f27f-473d-a324-e929c135fdd3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:32:28.651006 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a146c-f27f-473d-a324-e929c135fdd3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:28.652833 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a146c-f27f-473d-a324-e929c135fdd3 service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:28.663849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d4534-2b94-415e-afda-121ebd8aac7d service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:28.665058 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1397/2792] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:32:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:32:28.666539 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ac3d4534-2b94-415e-afda-121ebd8aac7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:32:28.687708 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d4534-2b94-415e-afda-121ebd8aac7d service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:28.696036 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fd445-bb59-4593-9439-a8e24f9a616c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:28.696731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fd445-bb59-4593-9439-a8e24f9a616c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:28.697188 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fd445-bb59-4593-9439-a8e24f9a616c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:28.697712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fd445-bb59-4593-9439-a8e24f9a616c admin admin] PATH_INFO: `/v3/users/66df80c268094c48abf73f1323aa23de` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:28.699265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9fd445-bb59-4593-9439-a8e24f9a616c admin admin] RBAC: Authorizing `identity:delete_user(user_id=66df80c268094c48abf73f1323aa23de)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:28.701637 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9fd445-bb59-4593-9439-a8e24f9a616c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:28.768595 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fd9fd445-bb59-4593-9439-a8e24f9a616c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66df80c268094c48abf73f1323aa23de'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:32:28.775513 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fd9fd445-bb59-4593-9439-a8e24f9a616c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66df80c268094c48abf73f1323aa23de'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:32:28.798316 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fd9fd445-bb59-4593-9439-a8e24f9a616c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66df80c268094c48abf73f1323aa23de'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:32:28.799542 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fd9fd445-bb59-4593-9439-a8e24f9a616c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66df80c268094c48abf73f1323aa23de'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:32:28.800571 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fd9fd445-bb59-4593-9439-a8e24f9a616c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66df80c268094c48abf73f1323aa23de'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:32:28.818283 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1396/2793] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:32:28 2022] DELETE /identity/v3/users/66df80c268094c48abf73f1323aa23de => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:32:28.995904 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a146c-f27f-473d-a324-e929c135fdd3 service placement] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:29.034271 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a146c-f27f-473d-a324-e929c135fdd3 service placement] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:29.092415 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b60f45-726a-48f4-9ee9-3a65c863cf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:29.093041 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b60f45-726a-48f4-9ee9-3a65c863cf19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:29.093871 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b60f45-726a-48f4-9ee9-3a65c863cf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:29.094151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b60f45-726a-48f4-9ee9-3a65c863cf19 admin admin] PATH_INFO: `/v3/projects/0679691b1c674dc4968638b8a93d4858` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:29.095286 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b60f45-726a-48f4-9ee9-3a65c863cf19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0679691b1c674dc4968638b8a93d4858)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:29.097177 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b60f45-726a-48f4-9ee9-3a65c863cf19 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:29.143708 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e2b60f45-726a-48f4-9ee9-3a65c863cf19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0679691b1c674dc4968638b8a93d4858', 'request_id': 'req-e2b60f45-726a-48f4-9ee9-3a65c863cf19'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:32:29.150051 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e2b60f45-726a-48f4-9ee9-3a65c863cf19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0679691b1c674dc4968638b8a93d4858', 'request_id': 'req-e2b60f45-726a-48f4-9ee9-3a65c863cf19'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:32:29.156436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e2b60f45-726a-48f4-9ee9-3a65c863cf19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0679691b1c674dc4968638b8a93d4858', 'request_id': 'req-e2b60f45-726a-48f4-9ee9-3a65c863cf19'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:32:29.169822 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e2b60f45-726a-48f4-9ee9-3a65c863cf19 admin admin] The token cache is being invalidate because project 0679691b1c674dc4968638b8a93d4858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:32:29.170055 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e2b60f45-726a-48f4-9ee9-3a65c863cf19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:32:29.170703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e2b60f45-726a-48f4-9ee9-3a65c863cf19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:32:29.172983 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1398/2794] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:32:28 2022] DELETE /identity/v3/projects/0679691b1c674dc4968638b8a93d4858 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:32:29.177925 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541ff88-63e1-4a0f-b4b5-9486df8265d7 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:29.178215 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541ff88-63e1-4a0f-b4b5-9486df8265d7 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:29.178495 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541ff88-63e1-4a0f-b4b5-9486df8265d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:29.198400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e541ff88-63e1-4a0f-b4b5-9486df8265d7 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:32:29.199657 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e541ff88-63e1-4a0f-b4b5-9486df8265d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:32:29.251551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:32:29.263743 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1397/2795] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:32:29.273152 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543415a-8fbb-45ba-8fd3-ef4f94e38d48 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:29.273436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543415a-8fbb-45ba-8fd3-ef4f94e38d48 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:29.273708 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543415a-8fbb-45ba-8fd3-ef4f94e38d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:29.296690 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-0543415a-8fbb-45ba-8fd3-ef4f94e38d48 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:32:29.297895 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0543415a-8fbb-45ba-8fd3-ef4f94e38d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:32:29.305755 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1399/2796] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:32:29.310195 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541ff88-63e1-4a0f-b4b5-9486df8265d7 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:29.320499 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541ff88-63e1-4a0f-b4b5-9486df8265d7 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:29.325096 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32efc89-6c8d-44c4-9d7b-7c5e9d33f0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:29.325695 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32efc89-6c8d-44c4-9d7b-7c5e9d33f0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:29.325978 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32efc89-6c8d-44c4-9d7b-7c5e9d33f0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:29.326254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32efc89-6c8d-44c4-9d7b-7c5e9d33f0e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:29.326934 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32efc89-6c8d-44c4-9d7b-7c5e9d33f0e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:32:29.327242 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32efc89-6c8d-44c4-9d7b-7c5e9d33f0e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:29.329383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32efc89-6c8d-44c4-9d7b-7c5e9d33f0e6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:29.341294 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1398/2797] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 16:32:29.346317 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543415a-8fbb-45ba-8fd3-ef4f94e38d48 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:29.355733 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543415a-8fbb-45ba-8fd3-ef4f94e38d48 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:29.360676 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e622bc-98fd-4088-9b9a-4ce55dafbd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:29.361273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e622bc-98fd-4088-9b9a-4ce55dafbd8f admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:29.361550 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e622bc-98fd-4088-9b9a-4ce55dafbd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:29.361825 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e622bc-98fd-4088-9b9a-4ce55dafbd8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:29.362493 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e622bc-98fd-4088-9b9a-4ce55dafbd8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:29.364418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e622bc-98fd-4088-9b9a-4ce55dafbd8f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:29.393574 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1400/2798] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:32:29 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:32:29.397725 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32efc89-6c8d-44c4-9d7b-7c5e9d33f0e6 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:29.408541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32efc89-6c8d-44c4-9d7b-7c5e9d33f0e6 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:29.465436 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b82d4a-b6a0-4bec-8659-de62d812fbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:29.465961 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b82d4a-b6a0-4bec-8659-de62d812fbd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:29.466242 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b82d4a-b6a0-4bec-8659-de62d812fbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:29.466516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b82d4a-b6a0-4bec-8659-de62d812fbd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:29.467185 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b82d4a-b6a0-4bec-8659-de62d812fbd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:29.469911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b82d4a-b6a0-4bec-8659-de62d812fbd6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:29.490568 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1399/2799] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:32:29 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:32:29.494776 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e622bc-98fd-4088-9b9a-4ce55dafbd8f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:29.505236 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e622bc-98fd-4088-9b9a-4ce55dafbd8f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:29.509922 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0f444-9bb1-4988-b70f-4d3ddaeac9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:29.510538 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0f444-9bb1-4988-b70f-4d3ddaeac9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:29.510925 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0f444-9bb1-4988-b70f-4d3ddaeac9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:29.511205 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0f444-9bb1-4988-b70f-4d3ddaeac9f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:29.511848 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0f444-9bb1-4988-b70f-4d3ddaeac9f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:29.513480 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0f444-9bb1-4988-b70f-4d3ddaeac9f4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:29.524903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1401/2800] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:32:29.528907 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b82d4a-b6a0-4bec-8659-de62d812fbd6 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:29.540574 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b82d4a-b6a0-4bec-8659-de62d812fbd6 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:29.545531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc9579-8d7e-402c-a0f9-e79d29898d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:29.546131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc9579-8d7e-402c-a0f9-e79d29898d5e admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:29.546449 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc9579-8d7e-402c-a0f9-e79d29898d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:29.547197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc9579-8d7e-402c-a0f9-e79d29898d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:29.547799 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc9579-8d7e-402c-a0f9-e79d29898d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:29.549407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc9579-8d7e-402c-a0f9-e79d29898d5e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:29.560789 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1400/2801] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:32:29.565650 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0f444-9bb1-4988-b70f-4d3ddaeac9f4 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:29.575978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0f444-9bb1-4988-b70f-4d3ddaeac9f4 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:29.581765 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13863b90-c6c7-4a7e-9f52-68919cd6286f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:29.582411 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13863b90-c6c7-4a7e-9f52-68919cd6286f admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:29.582715 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13863b90-c6c7-4a7e-9f52-68919cd6286f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:29.582993 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13863b90-c6c7-4a7e-9f52-68919cd6286f admin admin] PATH_INFO: `/v3/projects/999d0a4fb7cd46ca85bcd773ecba8695/users/04324489fbf24e3aaca409ebe7ab305f/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:29.595829 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:32:29.600953 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13863b90-c6c7-4a7e-9f52-68919cd6286f admin admin] RBAC: Authorizing `identity:create_grant(project_id=999d0a4fb7cd46ca85bcd773ecba8695, user_id=04324489fbf24e3aaca409ebe7ab305f, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:29.604311 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13863b90-c6c7-4a7e-9f52-68919cd6286f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:29.618515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1402/2802] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:32:29 2022] PUT /identity/v3/projects/999d0a4fb7cd46ca85bcd773ecba8695/users/04324489fbf24e3aaca409ebe7ab305f/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:32:29.630013 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fd95b-ae94-4eb9-934d-b8abdde2af2d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:29.630299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fd95b-ae94-4eb9-934d-b8abdde2af2d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:29.630574 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fd95b-ae94-4eb9-934d-b8abdde2af2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:29.661070 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-5b2fd95b-ae94-4eb9-934d-b8abdde2af2d None None] MFA Rules not processed for user `04324489fbf24e3aaca409ebe7ab305f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:32:29.662289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5b2fd95b-ae94-4eb9-934d-b8abdde2af2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:32:29.730507 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1401/2803] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:32:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:32:29.735184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13863b90-c6c7-4a7e-9f52-68919cd6286f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:29.796360 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13863b90-c6c7-4a7e-9f52-68919cd6286f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:29.801104 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f1bbc-15b7-4365-af4c-1433560cfda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:29.801700 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f1bbc-15b7-4365-af4c-1433560cfda1 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:29.801978 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f1bbc-15b7-4365-af4c-1433560cfda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:29.802297 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f1bbc-15b7-4365-af4c-1433560cfda1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:29.802920 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f1bbc-15b7-4365-af4c-1433560cfda1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:29.804822 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f1bbc-15b7-4365-af4c-1433560cfda1 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:29.836169 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1403/2804] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:32:29 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:32:29.837329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fd95b-ae94-4eb9-934d-b8abdde2af2d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:29.848236 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fd95b-ae94-4eb9-934d-b8abdde2af2d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:29.897271 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:32:29.911136 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71a068-0ade-4af6-b78c-dd28ffef63f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:29.911832 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71a068-0ade-4af6-b78c-dd28ffef63f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:29.912131 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71a068-0ade-4af6-b78c-dd28ffef63f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:29.912399 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71a068-0ade-4af6-b78c-dd28ffef63f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:29.913048 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71a068-0ade-4af6-b78c-dd28ffef63f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:29.915014 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71a068-0ade-4af6-b78c-dd28ffef63f0 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:29.943433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1402/2805] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:32:29 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:32:29.947479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f1bbc-15b7-4365-af4c-1433560cfda1 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:29.959026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f1bbc-15b7-4365-af4c-1433560cfda1 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:29.964181 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654fcc70-537d-4ec4-87b0-1c111ef00789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:29.964935 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fcc70-537d-4ec4-87b0-1c111ef00789 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:29.965261 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fcc70-537d-4ec4-87b0-1c111ef00789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:29.965574 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fcc70-537d-4ec4-87b0-1c111ef00789 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:29.966307 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654fcc70-537d-4ec4-87b0-1c111ef00789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:29.968035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654fcc70-537d-4ec4-87b0-1c111ef00789 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:29.979638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1404/2806] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:32:29.986133 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71a068-0ade-4af6-b78c-dd28ffef63f0 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:29.996264 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71a068-0ade-4af6-b78c-dd28ffef63f0 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:30.001475 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e60627-f8f2-4415-a1fe-ec879682219e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:30.002132 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e60627-f8f2-4415-a1fe-ec879682219e admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:30.002407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e60627-f8f2-4415-a1fe-ec879682219e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:30.002707 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e60627-f8f2-4415-a1fe-ec879682219e admin admin] PATH_INFO: `/v3/projects/f5dc757a20c640f1837a3c03c343ce6f/users/0ad3cf05ee1f482d83a07567de58bbcf/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:30.018712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e60627-f8f2-4415-a1fe-ec879682219e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5dc757a20c640f1837a3c03c343ce6f, user_id=0ad3cf05ee1f482d83a07567de58bbcf, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:30.021506 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e60627-f8f2-4415-a1fe-ec879682219e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:30.036754 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1403/2807] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:32:29 2022] PUT /identity/v3/projects/f5dc757a20c640f1837a3c03c343ce6f/users/0ad3cf05ee1f482d83a07567de58bbcf/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:32:30.040914 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654fcc70-537d-4ec4-87b0-1c111ef00789 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:30.051626 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654fcc70-537d-4ec4-87b0-1c111ef00789 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:30.109286 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da3fca-891f-4bf8-a8b7-483aa5820413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:30.109893 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da3fca-891f-4bf8-a8b7-483aa5820413 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:30.110171 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da3fca-891f-4bf8-a8b7-483aa5820413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:30.110446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da3fca-891f-4bf8-a8b7-483aa5820413 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:30.111129 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da3fca-891f-4bf8-a8b7-483aa5820413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:30.112730 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da3fca-891f-4bf8-a8b7-483aa5820413 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:30.122870 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1405/2808] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:32:30.127910 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e60627-f8f2-4415-a1fe-ec879682219e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:30.138313 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e60627-f8f2-4415-a1fe-ec879682219e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:30.144384 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1403e0-82da-40c6-b73b-b38f5d2de65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:30.145084 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1403e0-82da-40c6-b73b-b38f5d2de65b admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:30.145504 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1403e0-82da-40c6-b73b-b38f5d2de65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:30.145786 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1403e0-82da-40c6-b73b-b38f5d2de65b admin admin] PATH_INFO: `/v3/projects/f5dc757a20c640f1837a3c03c343ce6f/users/0ad3cf05ee1f482d83a07567de58bbcf/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:30.147900 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1403e0-82da-40c6-b73b-b38f5d2de65b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5dc757a20c640f1837a3c03c343ce6f, user_id=0ad3cf05ee1f482d83a07567de58bbcf, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:30.150545 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1403e0-82da-40c6-b73b-b38f5d2de65b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:30.162265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1404/2809] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:32:30 2022] PUT /identity/v3/projects/f5dc757a20c640f1837a3c03c343ce6f/users/0ad3cf05ee1f482d83a07567de58bbcf/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:32:30.174348 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66121e-cc3c-4d19-ba3a-69279dedc336 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:30.174630 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66121e-cc3c-4d19-ba3a-69279dedc336 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:30.174944 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66121e-cc3c-4d19-ba3a-69279dedc336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:30.204402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2a66121e-cc3c-4d19-ba3a-69279dedc336 None None] MFA Rules not processed for user `0ad3cf05ee1f482d83a07567de58bbcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:32:30.205441 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2a66121e-cc3c-4d19-ba3a-69279dedc336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:32:30.260949 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:32:30.288682 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1406/2810] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:32:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:32:30.299668 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1403e0-82da-40c6-b73b-b38f5d2de65b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:30.301390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6e1403e0-82da-40c6-b73b-b38f5d2de65b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:32:30.385434 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1403e0-82da-40c6-b73b-b38f5d2de65b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:30.389105 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cbc57-a10e-43d1-88cc-d6a28647a21a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:30.389712 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cbc57-a10e-43d1-88cc-d6a28647a21a service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:30.390125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cbc57-a10e-43d1-88cc-d6a28647a21a service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:30.390956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cbc57-a10e-43d1-88cc-d6a28647a21a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:30.450300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cbc57-a10e-43d1-88cc-d6a28647a21a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:30.451810 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cbc57-a10e-43d1-88cc-d6a28647a21a service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:30.462218 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1405/2811] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:32:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:32:32.419770 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66121e-cc3c-4d19-ba3a-69279dedc336 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:32.421899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2a66121e-cc3c-4d19-ba3a-69279dedc336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:32:32.434614 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66121e-cc3c-4d19-ba3a-69279dedc336 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:32.440268 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d14a0-74cc-479c-9a3c-0d0ccd7bb3a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:32.440875 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d14a0-74cc-479c-9a3c-0d0ccd7bb3a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:32.441151 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d14a0-74cc-479c-9a3c-0d0ccd7bb3a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:32.441418 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d14a0-74cc-479c-9a3c-0d0ccd7bb3a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:32.443178 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e36d14a0-74cc-479c-9a3c-0d0ccd7bb3a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:32:32.505092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d14a0-74cc-479c-9a3c-0d0ccd7bb3a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:32.506515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d14a0-74cc-479c-9a3c-0d0ccd7bb3a6 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:32.515798 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1407/2812] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:32:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:32:37.179920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cbc57-a10e-43d1-88cc-d6a28647a21a service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:37.189466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cbc57-a10e-43d1-88cc-d6a28647a21a service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:37.193275 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff62694-9df3-4c6e-8687-b1f53e3855df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:37.193879 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff62694-9df3-4c6e-8687-b1f53e3855df service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:37.194157 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff62694-9df3-4c6e-8687-b1f53e3855df service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:37.194428 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff62694-9df3-4c6e-8687-b1f53e3855df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:37.203976 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff62694-9df3-4c6e-8687-b1f53e3855df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:37.206017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff62694-9df3-4c6e-8687-b1f53e3855df service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:37.217012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1406/2813] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:32:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:32:42.511361 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=35 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:32:47.212414 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=81,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:32:52.009746 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354239f0-baa1-435b-9351-1325a8f73d5c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:52.010306 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354239f0-baa1-435b-9351-1325a8f73d5c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:52.010423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354239f0-baa1-435b-9351-1325a8f73d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:52.029962 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-354239f0-baa1-435b-9351-1325a8f73d5c None None] MFA Rules not processed for user `210d0f461d2c445a9688bc233d048079`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:32:52.031206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-354239f0-baa1-435b-9351-1325a8f73d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:32:52.038105 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1408/2814] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:32:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:32:52.377585 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff62694-9df3-4c6e-8687-b1f53e3855df service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:32:52.379664 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3ff62694-9df3-4c6e-8687-b1f53e3855df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:32:52.437383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff62694-9df3-4c6e-8687-b1f53e3855df service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:32:52.441536 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9174f-1646-4e4b-a0af-5ba409d3fa34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:32:52.442142 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9174f-1646-4e4b-a0af-5ba409d3fa34 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:32:52.442425 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9174f-1646-4e4b-a0af-5ba409d3fa34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:32:52.442721 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9174f-1646-4e4b-a0af-5ba409d3fa34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:32:52.444496 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bcd9174f-1646-4e4b-a0af-5ba409d3fa34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:32:52.454309 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9174f-1646-4e4b-a0af-5ba409d3fa34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:32:52.455965 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9174f-1646-4e4b-a0af-5ba409d3fa34 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:32:52.464356 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1407/2815] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:32:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:02.024783 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:02.460677 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=26 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:16.758816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354239f0-baa1-435b-9351-1325a8f73d5c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:16.821906 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354239f0-baa1-435b-9351-1325a8f73d5c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:16.826528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5644ef1-56e8-488b-bca2-b1969c0a6ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:16.827169 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5644ef1-56e8-488b-bca2-b1969c0a6ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:16.827445 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5644ef1-56e8-488b-bca2-b1969c0a6ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:16.827713 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5644ef1-56e8-488b-bca2-b1969c0a6ba6 admin admin] PATH_INFO: `/v3/users/04324489fbf24e3aaca409ebe7ab305f` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:16.828944 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5644ef1-56e8-488b-bca2-b1969c0a6ba6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04324489fbf24e3aaca409ebe7ab305f)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:16.831203 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5644ef1-56e8-488b-bca2-b1969c0a6ba6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:16.895582 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d5644ef1-56e8-488b-bca2-b1969c0a6ba6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04324489fbf24e3aaca409ebe7ab305f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:16.904313 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d5644ef1-56e8-488b-bca2-b1969c0a6ba6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04324489fbf24e3aaca409ebe7ab305f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:16.917077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d5644ef1-56e8-488b-bca2-b1969c0a6ba6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04324489fbf24e3aaca409ebe7ab305f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:16.917822 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d5644ef1-56e8-488b-bca2-b1969c0a6ba6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04324489fbf24e3aaca409ebe7ab305f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:16.925382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d5644ef1-56e8-488b-bca2-b1969c0a6ba6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04324489fbf24e3aaca409ebe7ab305f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:16.931540 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1409/2816] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:33:16 2022] DELETE /identity/v3/users/04324489fbf24e3aaca409ebe7ab305f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:16.935998 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67c334-3e8c-44b6-a3c5-4341459b6bc2 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:16.936492 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67c334-3e8c-44b6-a3c5-4341459b6bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:16.936624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67c334-3e8c-44b6-a3c5-4341459b6bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:16.955753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-7e67c334-3e8c-44b6-a3c5-4341459b6bc2 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:16.956961 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7e67c334-3e8c-44b6-a3c5-4341459b6bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:17.016610 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1408/2817] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:17.027396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5644ef1-56e8-488b-bca2-b1969c0a6ba6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:17.028317 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d5644ef1-56e8-488b-bca2-b1969c0a6ba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:17.085225 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5644ef1-56e8-488b-bca2-b1969c0a6ba6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:17.088900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27458fb-c3e8-4c95-972a-48fbea479916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:17.089494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27458fb-c3e8-4c95-972a-48fbea479916 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:17.089768 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27458fb-c3e8-4c95-972a-48fbea479916 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:17.090038 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27458fb-c3e8-4c95-972a-48fbea479916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:17.099750 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27458fb-c3e8-4c95-972a-48fbea479916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:17.101260 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27458fb-c3e8-4c95-972a-48fbea479916 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:17.110734 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1410/2818] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:17.402259 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67c334-3e8c-44b6-a3c5-4341459b6bc2 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:17.403882 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7e67c334-3e8c-44b6-a3c5-4341459b6bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:17.416765 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67c334-3e8c-44b6-a3c5-4341459b6bc2 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:17.422051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148eb47-7c31-490c-8853-d4389fc7e1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:17.422848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148eb47-7c31-490c-8853-d4389fc7e1e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:17.423187 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148eb47-7c31-490c-8853-d4389fc7e1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:17.423520 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148eb47-7c31-490c-8853-d4389fc7e1e1 admin admin] PATH_INFO: `/v3/projects/999d0a4fb7cd46ca85bcd773ecba8695` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:17.424660 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148eb47-7c31-490c-8853-d4389fc7e1e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=999d0a4fb7cd46ca85bcd773ecba8695)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:17.426753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148eb47-7c31-490c-8853-d4389fc7e1e1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:17.480834 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f148eb47-7c31-490c-8853-d4389fc7e1e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '999d0a4fb7cd46ca85bcd773ecba8695', 'request_id': 'req-f148eb47-7c31-490c-8853-d4389fc7e1e1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:17.488219 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f148eb47-7c31-490c-8853-d4389fc7e1e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '999d0a4fb7cd46ca85bcd773ecba8695', 'request_id': 'req-f148eb47-7c31-490c-8853-d4389fc7e1e1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:17.494330 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f148eb47-7c31-490c-8853-d4389fc7e1e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '999d0a4fb7cd46ca85bcd773ecba8695', 'request_id': 'req-f148eb47-7c31-490c-8853-d4389fc7e1e1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:17.506575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f148eb47-7c31-490c-8853-d4389fc7e1e1 admin admin] The token cache is being invalidate because project 999d0a4fb7cd46ca85bcd773ecba8695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:33:17.506892 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f148eb47-7c31-490c-8853-d4389fc7e1e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:17.507566 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f148eb47-7c31-490c-8853-d4389fc7e1e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:17.509077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1409/2819] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:33:17 2022] DELETE /identity/v3/projects/999d0a4fb7cd46ca85bcd773ecba8695 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:17.513098 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27458fb-c3e8-4c95-972a-48fbea479916 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:17.523461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27458fb-c3e8-4c95-972a-48fbea479916 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:17.541883 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:17.585465 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b335c2e-5462-4bda-9a28-2799998d00f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:17.586265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b335c2e-5462-4bda-9a28-2799998d00f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:17.586579 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b335c2e-5462-4bda-9a28-2799998d00f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:17.586882 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b335c2e-5462-4bda-9a28-2799998d00f8 admin admin] PATH_INFO: `/v3/users/0ad3cf05ee1f482d83a07567de58bbcf` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:17.587994 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b335c2e-5462-4bda-9a28-2799998d00f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ad3cf05ee1f482d83a07567de58bbcf)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:17.589888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b335c2e-5462-4bda-9a28-2799998d00f8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:17.638562 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9b335c2e-5462-4bda-9a28-2799998d00f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ad3cf05ee1f482d83a07567de58bbcf'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:17.645692 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9b335c2e-5462-4bda-9a28-2799998d00f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ad3cf05ee1f482d83a07567de58bbcf'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:17.657807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9b335c2e-5462-4bda-9a28-2799998d00f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ad3cf05ee1f482d83a07567de58bbcf'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:17.658508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9b335c2e-5462-4bda-9a28-2799998d00f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ad3cf05ee1f482d83a07567de58bbcf'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:17.665810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9b335c2e-5462-4bda-9a28-2799998d00f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ad3cf05ee1f482d83a07567de58bbcf'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:17.671903 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1411/2820] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:33:17 2022] DELETE /identity/v3/users/0ad3cf05ee1f482d83a07567de58bbcf => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:33:17.944680 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148eb47-7c31-490c-8853-d4389fc7e1e1 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:17.946154 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f148eb47-7c31-490c-8853-d4389fc7e1e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:17.960394 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148eb47-7c31-490c-8853-d4389fc7e1e1 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:18.020069 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa11485-36f5-42a1-8c5f-ff656355ef48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:18.020631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa11485-36f5-42a1-8c5f-ff656355ef48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:18.020951 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa11485-36f5-42a1-8c5f-ff656355ef48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:18.021231 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa11485-36f5-42a1-8c5f-ff656355ef48 admin admin] PATH_INFO: `/v3/projects/f5dc757a20c640f1837a3c03c343ce6f` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:18.022357 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa11485-36f5-42a1-8c5f-ff656355ef48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5dc757a20c640f1837a3c03c343ce6f)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:18.024300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa11485-36f5-42a1-8c5f-ff656355ef48 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:18.057435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:18.076109 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0fa11485-36f5-42a1-8c5f-ff656355ef48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5dc757a20c640f1837a3c03c343ce6f', 'request_id': 'req-0fa11485-36f5-42a1-8c5f-ff656355ef48'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:18.083753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0fa11485-36f5-42a1-8c5f-ff656355ef48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5dc757a20c640f1837a3c03c343ce6f', 'request_id': 'req-0fa11485-36f5-42a1-8c5f-ff656355ef48'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:18.089910 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0fa11485-36f5-42a1-8c5f-ff656355ef48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5dc757a20c640f1837a3c03c343ce6f', 'request_id': 'req-0fa11485-36f5-42a1-8c5f-ff656355ef48'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:18.101871 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0fa11485-36f5-42a1-8c5f-ff656355ef48 admin admin] The token cache is being invalidate because project f5dc757a20c640f1837a3c03c343ce6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:33:18.102157 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0fa11485-36f5-42a1-8c5f-ff656355ef48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:18.102828 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0fa11485-36f5-42a1-8c5f-ff656355ef48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:18.104312 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1410/2821] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:33:17 2022] DELETE /identity/v3/projects/f5dc757a20c640f1837a3c03c343ce6f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:18.109216 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1f95f-6634-409c-92ee-03fa47a5fb22 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:18.109734 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1f95f-6634-409c-92ee-03fa47a5fb22 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:18.109838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1f95f-6634-409c-92ee-03fa47a5fb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:18.130034 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-52d1f95f-6634-409c-92ee-03fa47a5fb22 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:18.131474 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-52d1f95f-6634-409c-92ee-03fa47a5fb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:18.196107 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1412/2822] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:18.204308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c0498-0d5f-42a9-b23b-14b008bcc86d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:18.204591 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c0498-0d5f-42a9-b23b-14b008bcc86d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:18.204918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c0498-0d5f-42a9-b23b-14b008bcc86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:18.354589 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-ba6c0498-0d5f-42a9-b23b-14b008bcc86d None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:18.355841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ba6c0498-0d5f-42a9-b23b-14b008bcc86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:18.364009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1411/2823] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:18.368458 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1f95f-6634-409c-92ee-03fa47a5fb22 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:18.378169 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1f95f-6634-409c-92ee-03fa47a5fb22 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:18.383388 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee1b18-d8f3-4e3a-9378-eed8442ea298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:18.383982 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee1b18-d8f3-4e3a-9378-eed8442ea298 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:18.384261 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee1b18-d8f3-4e3a-9378-eed8442ea298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:18.384537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee1b18-d8f3-4e3a-9378-eed8442ea298 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:18.385178 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee1b18-d8f3-4e3a-9378-eed8442ea298 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:33:18.385487 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee1b18-d8f3-4e3a-9378-eed8442ea298 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:18.387124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee1b18-d8f3-4e3a-9378-eed8442ea298 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:18.400690 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1413/2824] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:33:18.405360 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c0498-0d5f-42a9-b23b-14b008bcc86d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:18.416211 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c0498-0d5f-42a9-b23b-14b008bcc86d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:18.420977 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e86956-2afc-4236-b1a7-93c49d7fc6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:18.421575 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e86956-2afc-4236-b1a7-93c49d7fc6f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:18.421849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e86956-2afc-4236-b1a7-93c49d7fc6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:18.422116 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e86956-2afc-4236-b1a7-93c49d7fc6f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:18.422800 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e86956-2afc-4236-b1a7-93c49d7fc6f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:18.424691 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e86956-2afc-4236-b1a7-93c49d7fc6f4 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:18.451856 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1412/2825] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:33:18 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:33:18.456346 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee1b18-d8f3-4e3a-9378-eed8442ea298 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:18.467305 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee1b18-d8f3-4e3a-9378-eed8442ea298 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:18.493970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:18.528148 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28fc1d-1b5d-4af3-b0ea-ac6c0f8feec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:18.528777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28fc1d-1b5d-4af3-b0ea-ac6c0f8feec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:18.529053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28fc1d-1b5d-4af3-b0ea-ac6c0f8feec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:18.529354 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28fc1d-1b5d-4af3-b0ea-ac6c0f8feec3 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:18.529959 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28fc1d-1b5d-4af3-b0ea-ac6c0f8feec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:18.531982 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28fc1d-1b5d-4af3-b0ea-ac6c0f8feec3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:18.566423 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1414/2826] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:33:18 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:33:18.568339 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e86956-2afc-4236-b1a7-93c49d7fc6f4 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:18.579292 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e86956-2afc-4236-b1a7-93c49d7fc6f4 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:18.584484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324ce0a-d95a-47f5-917a-d8a3da599fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:18.585337 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324ce0a-d95a-47f5-917a-d8a3da599fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:18.585688 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324ce0a-d95a-47f5-917a-d8a3da599fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:18.586021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324ce0a-d95a-47f5-917a-d8a3da599fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:18.586767 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324ce0a-d95a-47f5-917a-d8a3da599fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:18.588368 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324ce0a-d95a-47f5-917a-d8a3da599fa4 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:18.604198 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1413/2827] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:33:18.605605 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28fc1d-1b5d-4af3-b0ea-ac6c0f8feec3 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:18.615865 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28fc1d-1b5d-4af3-b0ea-ac6c0f8feec3 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:18.620542 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a2499-30d4-4ad7-b26c-1ab54fabe1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:18.621152 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a2499-30d4-4ad7-b26c-1ab54fabe1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:18.621446 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a2499-30d4-4ad7-b26c-1ab54fabe1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:18.621738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a2499-30d4-4ad7-b26c-1ab54fabe1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:18.622387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a2499-30d4-4ad7-b26c-1ab54fabe1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:18.624019 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a2499-30d4-4ad7-b26c-1ab54fabe1a7 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:18.635044 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1415/2828] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:33:18.639204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324ce0a-d95a-47f5-917a-d8a3da599fa4 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:18.650435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324ce0a-d95a-47f5-917a-d8a3da599fa4 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:18.655709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010c6c1-7996-49cb-87d8-3a41034e359a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:18.656204 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010c6c1-7996-49cb-87d8-3a41034e359a admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:18.656484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010c6c1-7996-49cb-87d8-3a41034e359a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:18.656755 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010c6c1-7996-49cb-87d8-3a41034e359a admin admin] PATH_INFO: `/v3/projects/703567e8dacb42c280177e6be60e806b/users/8520ca94fb0443a3af0b6d7591aec947/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:18.672815 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010c6c1-7996-49cb-87d8-3a41034e359a admin admin] RBAC: Authorizing `identity:create_grant(project_id=703567e8dacb42c280177e6be60e806b, user_id=8520ca94fb0443a3af0b6d7591aec947, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:18.675495 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010c6c1-7996-49cb-87d8-3a41034e359a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:18.688714 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1414/2829] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:33:18 2022] PUT /identity/v3/projects/703567e8dacb42c280177e6be60e806b/users/8520ca94fb0443a3af0b6d7591aec947/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:18.701281 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b16ba7-38c8-4a2a-bef4-724d5ad07dda None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:18.701635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b16ba7-38c8-4a2a-bef4-724d5ad07dda None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:18.701921 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b16ba7-38c8-4a2a-bef4-724d5ad07dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:18.732519 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e3b16ba7-38c8-4a2a-bef4-724d5ad07dda None None] MFA Rules not processed for user `8520ca94fb0443a3af0b6d7591aec947`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:18.733787 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e3b16ba7-38c8-4a2a-bef4-724d5ad07dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:18.803309 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1416/2830] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:18.807918 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010c6c1-7996-49cb-87d8-3a41034e359a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:18.869564 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b16ba7-38c8-4a2a-bef4-724d5ad07dda None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:18.870970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e3b16ba7-38c8-4a2a-bef4-724d5ad07dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:18.875731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010c6c1-7996-49cb-87d8-3a41034e359a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:18.880449 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f62b55-93a3-43c1-8e51-63843330a221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:18.881128 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f62b55-93a3-43c1-8e51-63843330a221 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:18.881396 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f62b55-93a3-43c1-8e51-63843330a221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:18.881661 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f62b55-93a3-43c1-8e51-63843330a221 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:18.882274 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f62b55-93a3-43c1-8e51-63843330a221 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:18.884179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f62b55-93a3-43c1-8e51-63843330a221 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:18.915739 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1415/2831] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:33:18 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:33:18.916818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f62b55-93a3-43c1-8e51-63843330a221 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:18.933205 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b16ba7-38c8-4a2a-bef4-724d5ad07dda None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:18.956273 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:18.980717 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f62b55-93a3-43c1-8e51-63843330a221 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:18.985825 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cbe8dc-7036-4793-9872-d4863026d511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:18.986442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbe8dc-7036-4793-9872-d4863026d511 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:18.986782 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbe8dc-7036-4793-9872-d4863026d511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:18.986898 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5e8c9-a701-4ab5-b42c-b5475e408522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:18.987112 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbe8dc-7036-4793-9872-d4863026d511 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:18.987460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5e8c9-a701-4ab5-b42c-b5475e408522 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:18.987691 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cbe8dc-7036-4793-9872-d4863026d511 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:18.987809 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5e8c9-a701-4ab5-b42c-b5475e408522 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:18.987986 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5e8c9-a701-4ab5-b42c-b5475e408522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:18.989792 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-54c5e8c9-a701-4ab5-b42c-b5475e408522 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:18.989994 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cbe8dc-7036-4793-9872-d4863026d511 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:18.996287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:19.016876 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1416/2832] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:33:18 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:33:19.018177 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cbe8dc-7036-4793-9872-d4863026d511 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:19.029655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cbe8dc-7036-4793-9872-d4863026d511 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:19.034304 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db950c-f259-4228-929e-440f71128c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:19.034920 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db950c-f259-4228-929e-440f71128c8a admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:19.035203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db950c-f259-4228-929e-440f71128c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:19.035468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db950c-f259-4228-929e-440f71128c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:19.036124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db950c-f259-4228-929e-440f71128c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:19.038124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db950c-f259-4228-929e-440f71128c8a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:19.048551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1417/2833] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:33:19.053285 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db950c-f259-4228-929e-440f71128c8a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:19.059891 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5e8c9-a701-4ab5-b42c-b5475e408522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:19.061343 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5e8c9-a701-4ab5-b42c-b5475e408522 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:19.063878 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db950c-f259-4228-929e-440f71128c8a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:19.069339 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5e7ef-457f-4b14-8369-9a091edf42f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:19.069989 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5e7ef-457f-4b14-8369-9a091edf42f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:19.070266 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5e7ef-457f-4b14-8369-9a091edf42f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:19.070538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5e7ef-457f-4b14-8369-9a091edf42f8 admin admin] PATH_INFO: `/v3/projects/e47fd5a233504e21951e9668f358b513/users/d95328411b084587988d90ee4b22b8ca/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:19.071655 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1417/2834] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:33:19.089092 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5e7ef-457f-4b14-8369-9a091edf42f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e47fd5a233504e21951e9668f358b513, user_id=d95328411b084587988d90ee4b22b8ca, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:19.091886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5e7ef-457f-4b14-8369-9a091edf42f8 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:19.106313 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1418/2835] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:33:19 2022] PUT /identity/v3/projects/e47fd5a233504e21951e9668f358b513/users/d95328411b084587988d90ee4b22b8ca/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:33:19.110532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5e8c9-a701-4ab5-b42c-b5475e408522 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:19.121049 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5e8c9-a701-4ab5-b42c-b5475e408522 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:19.182718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066a510-c793-499e-938c-3e659b7c88a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:19.183332 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066a510-c793-499e-938c-3e659b7c88a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:19.183613 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066a510-c793-499e-938c-3e659b7c88a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:19.183911 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066a510-c793-499e-938c-3e659b7c88a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:19.184556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066a510-c793-499e-938c-3e659b7c88a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:19.186193 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066a510-c793-499e-938c-3e659b7c88a5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:19.197680 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1418/2836] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:33:19.201929 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5e7ef-457f-4b14-8369-9a091edf42f8 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:19.213353 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5e7ef-457f-4b14-8369-9a091edf42f8 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:19.218827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffa653-a6e1-4e74-9ffe-1b2986270105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:19.219564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffa653-a6e1-4e74-9ffe-1b2986270105 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:19.220551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffa653-a6e1-4e74-9ffe-1b2986270105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:19.220551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffa653-a6e1-4e74-9ffe-1b2986270105 admin admin] PATH_INFO: `/v3/projects/e47fd5a233504e21951e9668f358b513/users/d95328411b084587988d90ee4b22b8ca/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:19.222074 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fffa653-a6e1-4e74-9ffe-1b2986270105 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e47fd5a233504e21951e9668f358b513, user_id=d95328411b084587988d90ee4b22b8ca, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:19.224943 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fffa653-a6e1-4e74-9ffe-1b2986270105 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:19.235991 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1419/2837] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:33:19 2022] PUT /identity/v3/projects/e47fd5a233504e21951e9668f358b513/users/d95328411b084587988d90ee4b22b8ca/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:19.305586 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95e15e-00ab-46b0-ba79-b00318a83145 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:19.305868 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95e15e-00ab-46b0-ba79-b00318a83145 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:19.306135 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95e15e-00ab-46b0-ba79-b00318a83145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:19.337462 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4c95e15e-00ab-46b0-ba79-b00318a83145 None None] MFA Rules not processed for user `d95328411b084587988d90ee4b22b8ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:19.338759 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4c95e15e-00ab-46b0-ba79-b00318a83145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:19.413388 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1419/2838] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:19 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:33:19.425481 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffa653-a6e1-4e74-9ffe-1b2986270105 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:19.427844 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4fffa653-a6e1-4e74-9ffe-1b2986270105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:19.488424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffa653-a6e1-4e74-9ffe-1b2986270105 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:19.492849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca2c42-b2e1-49ed-a253-2f9269a956ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:19.493459 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca2c42-b2e1-49ed-a253-2f9269a956ea service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:19.493740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca2c42-b2e1-49ed-a253-2f9269a956ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:19.494021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca2c42-b2e1-49ed-a253-2f9269a956ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:19.504050 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca2c42-b2e1-49ed-a253-2f9269a956ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:19.506483 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca2c42-b2e1-49ed-a253-2f9269a956ea service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:19.516383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1420/2839] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:19.855816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95e15e-00ab-46b0-ba79-b00318a83145 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:19.866078 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95e15e-00ab-46b0-ba79-b00318a83145 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:19.871250 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f90fe-5467-4a3a-a450-cb49afddb3e7 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95328411b084587988d90ee4b22b8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e47fd5a233504e21951e9668f358b513', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:19.871873 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f90fe-5467-4a3a-a450-cb49afddb3e7 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:19.872157 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f90fe-5467-4a3a-a450-cb49afddb3e7 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:19.872593 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f90fe-5467-4a3a-a450-cb49afddb3e7 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:19.873272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f90fe-5467-4a3a-a450-cb49afddb3e7 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:33:19.873581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f90fe-5467-4a3a-a450-cb49afddb3e7 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:19.875231 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f90fe-5467-4a3a-a450-cb49afddb3e7 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:19.891385 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1420/2840] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:33:19.892587 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca2c42-b2e1-49ed-a253-2f9269a956ea service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:19.903429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca2c42-b2e1-49ed-a253-2f9269a956ea service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:19.908077 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8831fab-6e77-4f98-89bd-8414d1fa92e5 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95328411b084587988d90ee4b22b8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e47fd5a233504e21951e9668f358b513', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:19.908694 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8831fab-6e77-4f98-89bd-8414d1fa92e5 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:19.908970 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8831fab-6e77-4f98-89bd-8414d1fa92e5 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:19.909240 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8831fab-6e77-4f98-89bd-8414d1fa92e5 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:19.909911 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8831fab-6e77-4f98-89bd-8414d1fa92e5 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:19.912552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8831fab-6e77-4f98-89bd-8414d1fa92e5 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:19.944607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1421/2841] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:33:19 2022] POST /identity/v3/projects => generated 354 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:33:19.945996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f90fe-5467-4a3a-a450-cb49afddb3e7 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:19.956836 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f90fe-5467-4a3a-a450-cb49afddb3e7 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:19.966756 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:20.020270 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdaef7e-6ad5-4599-a9f8-2ac65cc71cd8 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95328411b084587988d90ee4b22b8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e47fd5a233504e21951e9668f358b513', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:20.021447 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaef7e-6ad5-4599-a9f8-2ac65cc71cd8 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:20.021936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaef7e-6ad5-4599-a9f8-2ac65cc71cd8 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:20.022322 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaef7e-6ad5-4599-a9f8-2ac65cc71cd8 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:20.023009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdaef7e-6ad5-4599-a9f8-2ac65cc71cd8 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:33:20.023314 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdaef7e-6ad5-4599-a9f8-2ac65cc71cd8 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:20.024910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdaef7e-6ad5-4599-a9f8-2ac65cc71cd8 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:20.038854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1421/2842] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:33:20.077033 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8831fab-6e77-4f98-89bd-8414d1fa92e5 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:20.088583 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8831fab-6e77-4f98-89bd-8414d1fa92e5 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:20.093531 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181d10b-1d5b-4c8d-9178-615a948914be tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95328411b084587988d90ee4b22b8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e47fd5a233504e21951e9668f358b513', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:20.094179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181d10b-1d5b-4c8d-9178-615a948914be tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:20.094465 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181d10b-1d5b-4c8d-9178-615a948914be tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:20.094776 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181d10b-1d5b-4c8d-9178-615a948914be tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] PATH_INFO: `/v3/projects/224bb0f45953467989c4d339e7860332` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:20.095934 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181d10b-1d5b-4c8d-9178-615a948914be tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] RBAC: Authorizing `identity:delete_project(project_id=224bb0f45953467989c4d339e7860332)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:20.097839 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181d10b-1d5b-4c8d-9178-615a948914be tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:20.146979 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4181d10b-1d5b-4c8d-9178-615a948914be tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '224bb0f45953467989c4d339e7860332', 'request_id': 'req-4181d10b-1d5b-4c8d-9178-615a948914be'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:20.149189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:INSERT=7,keystone:DELETE=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:20.155069 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4181d10b-1d5b-4c8d-9178-615a948914be tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '224bb0f45953467989c4d339e7860332', 'request_id': 'req-4181d10b-1d5b-4c8d-9178-615a948914be'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:20.163027 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4181d10b-1d5b-4c8d-9178-615a948914be tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '224bb0f45953467989c4d339e7860332', 'request_id': 'req-4181d10b-1d5b-4c8d-9178-615a948914be'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:20.175360 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4181d10b-1d5b-4c8d-9178-615a948914be tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] The token cache is being invalidate because project 224bb0f45953467989c4d339e7860332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:33:20.175651 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4181d10b-1d5b-4c8d-9178-615a948914be tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:20.176326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4181d10b-1d5b-4c8d-9178-615a948914be tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:20.177830 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1422/2843] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:33:20 2022] DELETE /identity/v3/projects/224bb0f45953467989c4d339e7860332 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:20.392508 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdaef7e-6ad5-4599-a9f8-2ac65cc71cd8 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:20.394274 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5cdaef7e-6ad5-4599-a9f8-2ac65cc71cd8 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:20.464036 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdaef7e-6ad5-4599-a9f8-2ac65cc71cd8 tempest-VolumeQuotaClassesTest-144754243 tempest-VolumeQuotaClassesTest-144754243-project-admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:20.524062 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18aa8c-8662-46f2-8921-d7e26c7e8b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:20.524853 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18aa8c-8662-46f2-8921-d7e26c7e8b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:20.525140 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18aa8c-8662-46f2-8921-d7e26c7e8b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:20.525485 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18aa8c-8662-46f2-8921-d7e26c7e8b7b admin admin] PATH_INFO: `/v3/users/8520ca94fb0443a3af0b6d7591aec947` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:20.526855 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18aa8c-8662-46f2-8921-d7e26c7e8b7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8520ca94fb0443a3af0b6d7591aec947)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:20.529004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18aa8c-8662-46f2-8921-d7e26c7e8b7b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:20.588795 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ae18aa8c-8662-46f2-8921-d7e26c7e8b7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8520ca94fb0443a3af0b6d7591aec947'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:20.595539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:20.598073 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ae18aa8c-8662-46f2-8921-d7e26c7e8b7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8520ca94fb0443a3af0b6d7591aec947'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:20.614649 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ae18aa8c-8662-46f2-8921-d7e26c7e8b7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8520ca94fb0443a3af0b6d7591aec947'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:20.616376 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ae18aa8c-8662-46f2-8921-d7e26c7e8b7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8520ca94fb0443a3af0b6d7591aec947'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:20.624509 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ae18aa8c-8662-46f2-8921-d7e26c7e8b7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8520ca94fb0443a3af0b6d7591aec947'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:20.631120 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1422/2844] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:33:20 2022] DELETE /identity/v3/users/8520ca94fb0443a3af0b6d7591aec947 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:20.635552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6d86c-e2c1-4d6d-a19f-3744efb4e85c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:20.635836 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6d86c-e2c1-4d6d-a19f-3744efb4e85c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:20.636115 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6d86c-e2c1-4d6d-a19f-3744efb4e85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:20.656435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-34c6d86c-e2c1-4d6d-a19f-3744efb4e85c None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:20.657781 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-34c6d86c-e2c1-4d6d-a19f-3744efb4e85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:20.718713 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1423/2845] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:20.729167 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18aa8c-8662-46f2-8921-d7e26c7e8b7b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:20.730575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ae18aa8c-8662-46f2-8921-d7e26c7e8b7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:20.790301 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18aa8c-8662-46f2-8921-d7e26c7e8b7b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:20.793977 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe5829-c780-4559-9865-349e8d8ef237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:20.794858 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe5829-c780-4559-9865-349e8d8ef237 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:20.795139 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe5829-c780-4559-9865-349e8d8ef237 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:20.795411 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe5829-c780-4559-9865-349e8d8ef237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:20.804769 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe5829-c780-4559-9865-349e8d8ef237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:20.806209 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe5829-c780-4559-9865-349e8d8ef237 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:20.816092 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1423/2846] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:21.111654 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6d86c-e2c1-4d6d-a19f-3744efb4e85c None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:21.113203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-34c6d86c-e2c1-4d6d-a19f-3744efb4e85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:21.137606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6d86c-e2c1-4d6d-a19f-3744efb4e85c None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:21.144164 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008cdba-34a6-4363-ba55-fd22d6de5aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:21.144956 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008cdba-34a6-4363-ba55-fd22d6de5aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:21.145373 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008cdba-34a6-4363-ba55-fd22d6de5aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:21.146302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008cdba-34a6-4363-ba55-fd22d6de5aba admin admin] PATH_INFO: `/v3/projects/703567e8dacb42c280177e6be60e806b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:21.148178 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008cdba-34a6-4363-ba55-fd22d6de5aba admin admin] RBAC: Authorizing `identity:delete_project(project_id=703567e8dacb42c280177e6be60e806b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:21.151044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008cdba-34a6-4363-ba55-fd22d6de5aba admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:21.205913 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0008cdba-34a6-4363-ba55-fd22d6de5aba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '703567e8dacb42c280177e6be60e806b', 'request_id': 'req-0008cdba-34a6-4363-ba55-fd22d6de5aba'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:21.217733 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0008cdba-34a6-4363-ba55-fd22d6de5aba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '703567e8dacb42c280177e6be60e806b', 'request_id': 'req-0008cdba-34a6-4363-ba55-fd22d6de5aba'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:21.229016 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0008cdba-34a6-4363-ba55-fd22d6de5aba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '703567e8dacb42c280177e6be60e806b', 'request_id': 'req-0008cdba-34a6-4363-ba55-fd22d6de5aba'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:21.241899 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0008cdba-34a6-4363-ba55-fd22d6de5aba admin admin] The token cache is being invalidate because project 703567e8dacb42c280177e6be60e806b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:33:21.242189 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0008cdba-34a6-4363-ba55-fd22d6de5aba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:21.242880 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0008cdba-34a6-4363-ba55-fd22d6de5aba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:21.244447 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1424/2847] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:33:21 2022] DELETE /identity/v3/projects/703567e8dacb42c280177e6be60e806b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:21.248491 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe5829-c780-4559-9865-349e8d8ef237 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:21.258842 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe5829-c780-4559-9865-349e8d8ef237 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:21.331035 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9073e-cbb0-4010-aebf-6d4376b447db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:21.331833 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9073e-cbb0-4010-aebf-6d4376b447db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:21.332214 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9073e-cbb0-4010-aebf-6d4376b447db admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:21.332587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9073e-cbb0-4010-aebf-6d4376b447db admin admin] PATH_INFO: `/v3/users/d95328411b084587988d90ee4b22b8ca` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:21.335524 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9073e-cbb0-4010-aebf-6d4376b447db admin admin] RBAC: Authorizing `identity:delete_user(user_id=d95328411b084587988d90ee4b22b8ca)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:21.337835 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9073e-cbb0-4010-aebf-6d4376b447db admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:21.397469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b2f9073e-cbb0-4010-aebf-6d4376b447db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd95328411b084587988d90ee4b22b8ca'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:21.405016 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b2f9073e-cbb0-4010-aebf-6d4376b447db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd95328411b084587988d90ee4b22b8ca'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:21.421486 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b2f9073e-cbb0-4010-aebf-6d4376b447db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd95328411b084587988d90ee4b22b8ca'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:21.422244 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b2f9073e-cbb0-4010-aebf-6d4376b447db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd95328411b084587988d90ee4b22b8ca'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:21.425411 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:21.432194 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b2f9073e-cbb0-4010-aebf-6d4376b447db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd95328411b084587988d90ee4b22b8ca'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:21.442764 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1424/2848] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:33:21 2022] DELETE /identity/v3/users/d95328411b084587988d90ee4b22b8ca => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:33:21.746841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008cdba-34a6-4363-ba55-fd22d6de5aba admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:21.748610 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0008cdba-34a6-4363-ba55-fd22d6de5aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:21.768003 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008cdba-34a6-4363-ba55-fd22d6de5aba admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:21.833728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef2ade-c395-498f-80ea-4bc07758e810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:21.834374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef2ade-c395-498f-80ea-4bc07758e810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:21.834646 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef2ade-c395-498f-80ea-4bc07758e810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:21.835197 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef2ade-c395-498f-80ea-4bc07758e810 admin admin] PATH_INFO: `/v3/projects/e47fd5a233504e21951e9668f358b513` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:21.836348 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef2ade-c395-498f-80ea-4bc07758e810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e47fd5a233504e21951e9668f358b513)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:21.838234 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef2ade-c395-498f-80ea-4bc07758e810 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:21.845006 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:21.890016 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4aef2ade-c395-498f-80ea-4bc07758e810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e47fd5a233504e21951e9668f358b513', 'request_id': 'req-4aef2ade-c395-498f-80ea-4bc07758e810'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:21.898279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4aef2ade-c395-498f-80ea-4bc07758e810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e47fd5a233504e21951e9668f358b513', 'request_id': 'req-4aef2ade-c395-498f-80ea-4bc07758e810'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:21.904435 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4aef2ade-c395-498f-80ea-4bc07758e810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e47fd5a233504e21951e9668f358b513', 'request_id': 'req-4aef2ade-c395-498f-80ea-4bc07758e810'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:21.917693 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4aef2ade-c395-498f-80ea-4bc07758e810 admin admin] The token cache is being invalidate because project e47fd5a233504e21951e9668f358b513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:33:21.918007 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4aef2ade-c395-498f-80ea-4bc07758e810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:21.918625 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4aef2ade-c395-498f-80ea-4bc07758e810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:21.920128 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1425/2849] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:33:21 2022] DELETE /identity/v3/projects/e47fd5a233504e21951e9668f358b513 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:21.925124 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425cda7-f33c-4cda-88fe-834ebb599dbb None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:21.925413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425cda7-f33c-4cda-88fe-834ebb599dbb None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:21.925702 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425cda7-f33c-4cda-88fe-834ebb599dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:21.961153 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4425cda7-f33c-4cda-88fe-834ebb599dbb None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:21.962407 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4425cda7-f33c-4cda-88fe-834ebb599dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:22.029440 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1425/2850] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:22.037185 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c32b3-8c87-4bad-aab4-3f57c0c6d5b3 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:22.037466 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c32b3-8c87-4bad-aab4-3f57c0c6d5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:22.037746 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c32b3-8c87-4bad-aab4-3f57c0c6d5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:22.058160 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d77c32b3-8c87-4bad-aab4-3f57c0c6d5b3 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:22.059420 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d77c32b3-8c87-4bad-aab4-3f57c0c6d5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:22.068553 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1426/2851] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:22.073176 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425cda7-f33c-4cda-88fe-834ebb599dbb None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:22.083037 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425cda7-f33c-4cda-88fe-834ebb599dbb None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:22.088540 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8893f97-20d5-4bca-8f0b-4786de859ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:22.089150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8893f97-20d5-4bca-8f0b-4786de859ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:22.089457 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8893f97-20d5-4bca-8f0b-4786de859ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:22.089728 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8893f97-20d5-4bca-8f0b-4786de859ce7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:22.090380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8893f97-20d5-4bca-8f0b-4786de859ce7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:33:22.090710 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8893f97-20d5-4bca-8f0b-4786de859ce7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:22.092341 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8893f97-20d5-4bca-8f0b-4786de859ce7 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:22.104636 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1426/2852] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:33:22.109144 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c32b3-8c87-4bad-aab4-3f57c0c6d5b3 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:22.119352 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c32b3-8c87-4bad-aab4-3f57c0c6d5b3 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:22.124174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98275b0-6a66-4e29-8b21-4809bc700953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:22.124849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98275b0-6a66-4e29-8b21-4809bc700953 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:22.125196 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98275b0-6a66-4e29-8b21-4809bc700953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:22.125538 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98275b0-6a66-4e29-8b21-4809bc700953 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:22.126262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98275b0-6a66-4e29-8b21-4809bc700953 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:22.128300 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98275b0-6a66-4e29-8b21-4809bc700953 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:22.160135 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1427/2853] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:33:22 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:33:22.164445 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8893f97-20d5-4bca-8f0b-4786de859ce7 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:22.174556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8893f97-20d5-4bca-8f0b-4786de859ce7 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:22.232065 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cff1fa-063c-4d29-b2d7-1866281a13e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:22.232655 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cff1fa-063c-4d29-b2d7-1866281a13e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:22.232924 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cff1fa-063c-4d29-b2d7-1866281a13e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:22.233192 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cff1fa-063c-4d29-b2d7-1866281a13e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:22.233821 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cff1fa-063c-4d29-b2d7-1866281a13e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:22.235752 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cff1fa-063c-4d29-b2d7-1866281a13e9 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:22.258208 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1427/2854] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:33:22 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:33:22.262329 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98275b0-6a66-4e29-8b21-4809bc700953 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:22.272576 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98275b0-6a66-4e29-8b21-4809bc700953 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:22.277021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b825046-4892-4a80-82ad-8136b07ca720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:22.277650 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b825046-4892-4a80-82ad-8136b07ca720 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:22.277926 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b825046-4892-4a80-82ad-8136b07ca720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:22.278192 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b825046-4892-4a80-82ad-8136b07ca720 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:22.278844 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b825046-4892-4a80-82ad-8136b07ca720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:22.280438 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b825046-4892-4a80-82ad-8136b07ca720 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:22.290278 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1428/2855] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:33:22.294148 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cff1fa-063c-4d29-b2d7-1866281a13e9 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:22.308829 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cff1fa-063c-4d29-b2d7-1866281a13e9 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:22.313681 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b79212-12c8-4c5a-84bf-e13e0c435653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:22.314328 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b79212-12c8-4c5a-84bf-e13e0c435653 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:22.314607 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b79212-12c8-4c5a-84bf-e13e0c435653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:22.314911 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b79212-12c8-4c5a-84bf-e13e0c435653 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:22.315556 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b79212-12c8-4c5a-84bf-e13e0c435653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:22.317180 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b79212-12c8-4c5a-84bf-e13e0c435653 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:22.329287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1428/2856] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:33:22.334043 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b825046-4892-4a80-82ad-8136b07ca720 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:22.344136 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b825046-4892-4a80-82ad-8136b07ca720 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:22.348588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79fafe-5ab4-48f6-9aeb-9c151798659a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:22.349327 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79fafe-5ab4-48f6-9aeb-9c151798659a admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:22.349609 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79fafe-5ab4-48f6-9aeb-9c151798659a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:22.349870 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79fafe-5ab4-48f6-9aeb-9c151798659a admin admin] PATH_INFO: `/v3/projects/ee77aa448bb74afb9f9377fab9d112d9/users/94e720e68c3e4218b86ddd18d7d94028/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:22.364187 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79fafe-5ab4-48f6-9aeb-9c151798659a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee77aa448bb74afb9f9377fab9d112d9, user_id=94e720e68c3e4218b86ddd18d7d94028, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:22.367344 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79fafe-5ab4-48f6-9aeb-9c151798659a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:22.380000 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1429/2857] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:33:22 2022] PUT /identity/v3/projects/ee77aa448bb74afb9f9377fab9d112d9/users/94e720e68c3e4218b86ddd18d7d94028/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:22.392275 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f45df-3981-4b88-8ba5-08b1403122c1 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:22.392563 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f45df-3981-4b88-8ba5-08b1403122c1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:22.392834 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f45df-3981-4b88-8ba5-08b1403122c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:22.422837 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-fd1f45df-3981-4b88-8ba5-08b1403122c1 None None] MFA Rules not processed for user `94e720e68c3e4218b86ddd18d7d94028`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:22.424048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fd1f45df-3981-4b88-8ba5-08b1403122c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:22.472436 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:22.492660 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1429/2858] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:33:22.497223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79fafe-5ab4-48f6-9aeb-9c151798659a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:22.557636 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79fafe-5ab4-48f6-9aeb-9c151798659a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:22.562150 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060d8cd-3a53-4e04-be1e-ab95e4ac7ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:22.562772 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060d8cd-3a53-4e04-be1e-ab95e4ac7ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:22.563052 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060d8cd-3a53-4e04-be1e-ab95e4ac7ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:22.563321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060d8cd-3a53-4e04-be1e-ab95e4ac7ec6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:22.563959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060d8cd-3a53-4e04-be1e-ab95e4ac7ec6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:22.565850 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060d8cd-3a53-4e04-be1e-ab95e4ac7ec6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:22.594044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1430/2859] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:33:22 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:33:22.600894 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f45df-3981-4b88-8ba5-08b1403122c1 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:22.611590 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f45df-3981-4b88-8ba5-08b1403122c1 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:22.668774 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a80bb-bec2-43f4-a02b-1694372d46d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:22.669386 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a80bb-bec2-43f4-a02b-1694372d46d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:22.669668 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a80bb-bec2-43f4-a02b-1694372d46d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:22.669941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a80bb-bec2-43f4-a02b-1694372d46d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:22.670571 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a80bb-bec2-43f4-a02b-1694372d46d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:22.672520 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a80bb-bec2-43f4-a02b-1694372d46d9 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:22.694160 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1430/2860] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:33:22 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:33:22.698373 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060d8cd-3a53-4e04-be1e-ab95e4ac7ec6 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:22.710081 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060d8cd-3a53-4e04-be1e-ab95e4ac7ec6 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:22.714955 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8875998-05e0-4875-8fd4-43821023aa3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:22.715642 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8875998-05e0-4875-8fd4-43821023aa3b admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:22.715923 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8875998-05e0-4875-8fd4-43821023aa3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:22.716198 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8875998-05e0-4875-8fd4-43821023aa3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:22.716842 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8875998-05e0-4875-8fd4-43821023aa3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:22.718489 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8875998-05e0-4875-8fd4-43821023aa3b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:22.729166 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1431/2861] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:33:22.733325 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a80bb-bec2-43f4-a02b-1694372d46d9 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:22.743558 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a80bb-bec2-43f4-a02b-1694372d46d9 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:22.748291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5df562-598b-4e5c-813e-6c778175b5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:22.748936 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5df562-598b-4e5c-813e-6c778175b5f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:22.749219 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5df562-598b-4e5c-813e-6c778175b5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:22.749498 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5df562-598b-4e5c-813e-6c778175b5f4 admin admin] PATH_INFO: `/v3/projects/acb811220e8b4d788b628e0aea182921/users/204fd056c9e54678adeb14f85a56f005/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:22.764858 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5df562-598b-4e5c-813e-6c778175b5f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb811220e8b4d788b628e0aea182921, user_id=204fd056c9e54678adeb14f85a56f005, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:22.767565 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5df562-598b-4e5c-813e-6c778175b5f4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:22.782040 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1431/2862] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:33:22 2022] PUT /identity/v3/projects/acb811220e8b4d788b628e0aea182921/users/204fd056c9e54678adeb14f85a56f005/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:22.785946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8875998-05e0-4875-8fd4-43821023aa3b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:22.792897 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:22.797938 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8875998-05e0-4875-8fd4-43821023aa3b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:22.856841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9e66a-60a8-4df7-be29-a0d246eab76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:22.857443 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9e66a-60a8-4df7-be29-a0d246eab76e admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:22.857718 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9e66a-60a8-4df7-be29-a0d246eab76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:22.857987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9e66a-60a8-4df7-be29-a0d246eab76e admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:22.858612 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9e66a-60a8-4df7-be29-a0d246eab76e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:22.860215 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9e66a-60a8-4df7-be29-a0d246eab76e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:22.870565 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1432/2863] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:33:22.874791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5df562-598b-4e5c-813e-6c778175b5f4 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:22.885976 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5df562-598b-4e5c-813e-6c778175b5f4 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:22.890599 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061941d6-dd68-43fe-a155-3aa383fad970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:22.891287 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061941d6-dd68-43fe-a155-3aa383fad970 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:22.891601 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061941d6-dd68-43fe-a155-3aa383fad970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:22.891866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061941d6-dd68-43fe-a155-3aa383fad970 admin admin] PATH_INFO: `/v3/projects/acb811220e8b4d788b628e0aea182921/users/204fd056c9e54678adeb14f85a56f005/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:22.893791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061941d6-dd68-43fe-a155-3aa383fad970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb811220e8b4d788b628e0aea182921, user_id=204fd056c9e54678adeb14f85a56f005, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:22.896777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061941d6-dd68-43fe-a155-3aa383fad970 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:22.908308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1432/2864] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:33:22 2022] PUT /identity/v3/projects/acb811220e8b4d788b628e0aea182921/users/204fd056c9e54678adeb14f85a56f005/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:22.918631 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3f025-668f-40dd-916a-0ba5912fd843 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:22.918950 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3f025-668f-40dd-916a-0ba5912fd843 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:22.919225 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3f025-668f-40dd-916a-0ba5912fd843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:22.947768 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-05b3f025-668f-40dd-916a-0ba5912fd843 None None] MFA Rules not processed for user `204fd056c9e54678adeb14f85a56f005`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:22.948971 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-05b3f025-668f-40dd-916a-0ba5912fd843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:23.019199 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1433/2865] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:22 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:23.031637 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061941d6-dd68-43fe-a155-3aa383fad970 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:23.033371 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-061941d6-dd68-43fe-a155-3aa383fad970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:23.093834 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061941d6-dd68-43fe-a155-3aa383fad970 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:23.097575 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037216b-b99d-459d-9a41-a6aedc8c9fb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:23.098169 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037216b-b99d-459d-9a41-a6aedc8c9fb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:23.098442 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037216b-b99d-459d-9a41-a6aedc8c9fb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:23.098790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037216b-b99d-459d-9a41-a6aedc8c9fb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:23.154304 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:23.156587 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037216b-b99d-459d-9a41-a6aedc8c9fb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:23.158900 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037216b-b99d-459d-9a41-a6aedc8c9fb9 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:23.169117 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1433/2866] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:27.410392 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3f025-668f-40dd-916a-0ba5912fd843 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:27.411874 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-05b3f025-668f-40dd-916a-0ba5912fd843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:27.424696 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3f025-668f-40dd-916a-0ba5912fd843 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:27.428365 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007cba55-bc49-41a6-b536-149aa942a2de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:27.429710 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007cba55-bc49-41a6-b536-149aa942a2de service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:27.429847 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007cba55-bc49-41a6-b536-149aa942a2de service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:27.430004 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007cba55-bc49-41a6-b536-149aa942a2de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:27.439175 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007cba55-bc49-41a6-b536-149aa942a2de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:27.440716 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007cba55-bc49-41a6-b536-149aa942a2de service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:27.450887 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1434/2867] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:31.394420 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037216b-b99d-459d-9a41-a6aedc8c9fb9 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:31.395889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1037216b-b99d-459d-9a41-a6aedc8c9fb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:31.492910 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037216b-b99d-459d-9a41-a6aedc8c9fb9 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:31.497953 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bb651-a5ce-4c65-b3f2-f438eb34409c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034d75ab747e4c0cab040e47c6a6236e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:31.499026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bb651-a5ce-4c65-b3f2-f438eb34409c service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:31.499026 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bb651-a5ce-4c65-b3f2-f438eb34409c service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:31.499272 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bb651-a5ce-4c65-b3f2-f438eb34409c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:31.500954 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c07bb651-a5ce-4c65-b3f2-f438eb34409c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:31.513665 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07bb651-a5ce-4c65-b3f2-f438eb34409c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:31.515311 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07bb651-a5ce-4c65-b3f2-f438eb34409c service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:31.525572 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1434/2868] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:32.363878 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007cba55-bc49-41a6-b536-149aa942a2de service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:32.426896 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007cba55-bc49-41a6-b536-149aa942a2de service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:32.431607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9118304-6528-400a-b6a6-ee30f118f876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:32.432226 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9118304-6528-400a-b6a6-ee30f118f876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:32.432501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9118304-6528-400a-b6a6-ee30f118f876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:32.432771 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9118304-6528-400a-b6a6-ee30f118f876 admin admin] PATH_INFO: `/v3/users/94e720e68c3e4218b86ddd18d7d94028` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:32.433904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9118304-6528-400a-b6a6-ee30f118f876 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94e720e68c3e4218b86ddd18d7d94028)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:32.435848 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9118304-6528-400a-b6a6-ee30f118f876 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:32.449987 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:32.483800 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e9118304-6528-400a-b6a6-ee30f118f876 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94e720e68c3e4218b86ddd18d7d94028'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:32.490740 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e9118304-6528-400a-b6a6-ee30f118f876 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94e720e68c3e4218b86ddd18d7d94028'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:32.503113 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e9118304-6528-400a-b6a6-ee30f118f876 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94e720e68c3e4218b86ddd18d7d94028'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:32.503793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e9118304-6528-400a-b6a6-ee30f118f876 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94e720e68c3e4218b86ddd18d7d94028'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:32.504397 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e9118304-6528-400a-b6a6-ee30f118f876 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94e720e68c3e4218b86ddd18d7d94028'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:32.517222 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1435/2869] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:33:32 2022] DELETE /identity/v3/users/94e720e68c3e4218b86ddd18d7d94028 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:32.521777 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615845f-43b1-448a-98d2-4c6518efe605 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:32.522316 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615845f-43b1-448a-98d2-4c6518efe605 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:32.522452 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615845f-43b1-448a-98d2-4c6518efe605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:32.542411 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4615845f-43b1-448a-98d2-4c6518efe605 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:32.543657 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4615845f-43b1-448a-98d2-4c6518efe605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:32.603357 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1435/2870] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:32.613444 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9118304-6528-400a-b6a6-ee30f118f876 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:32.614922 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e9118304-6528-400a-b6a6-ee30f118f876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:32.672818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9118304-6528-400a-b6a6-ee30f118f876 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:32.676491 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c094c6-4f81-4de8-a568-d3c04e94eab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:32.677101 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c094c6-4f81-4de8-a568-d3c04e94eab2 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:32.677378 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c094c6-4f81-4de8-a568-d3c04e94eab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:32.677654 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c094c6-4f81-4de8-a568-d3c04e94eab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:32.687737 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c094c6-4f81-4de8-a568-d3c04e94eab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:32.689286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c094c6-4f81-4de8-a568-d3c04e94eab2 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:32.699400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1436/2871] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:32.968627 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615845f-43b1-448a-98d2-4c6518efe605 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:32.970125 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4615845f-43b1-448a-98d2-4c6518efe605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:32.984614 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615845f-43b1-448a-98d2-4c6518efe605 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:32.990120 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b6c48-0560-41a0-9532-d287f0da781a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:32.990820 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b6c48-0560-41a0-9532-d287f0da781a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:32.991084 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b6c48-0560-41a0-9532-d287f0da781a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:32.991483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b6c48-0560-41a0-9532-d287f0da781a admin admin] PATH_INFO: `/v3/projects/ee77aa448bb74afb9f9377fab9d112d9` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:32.992494 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b6c48-0560-41a0-9532-d287f0da781a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee77aa448bb74afb9f9377fab9d112d9)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:32.995277 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b6c48-0560-41a0-9532-d287f0da781a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:33.044861 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-400b6c48-0560-41a0-9532-d287f0da781a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee77aa448bb74afb9f9377fab9d112d9', 'request_id': 'req-400b6c48-0560-41a0-9532-d287f0da781a'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:33.051053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-400b6c48-0560-41a0-9532-d287f0da781a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee77aa448bb74afb9f9377fab9d112d9', 'request_id': 'req-400b6c48-0560-41a0-9532-d287f0da781a'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:33.055619 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:33.058163 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-400b6c48-0560-41a0-9532-d287f0da781a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee77aa448bb74afb9f9377fab9d112d9', 'request_id': 'req-400b6c48-0560-41a0-9532-d287f0da781a'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:33.071655 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-400b6c48-0560-41a0-9532-d287f0da781a admin admin] The token cache is being invalidate because project ee77aa448bb74afb9f9377fab9d112d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:33:33.071996 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-400b6c48-0560-41a0-9532-d287f0da781a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:33.072620 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-400b6c48-0560-41a0-9532-d287f0da781a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:33.074096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1436/2872] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:33:32 2022] DELETE /identity/v3/projects/ee77aa448bb74afb9f9377fab9d112d9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:33.078269 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c094c6-4f81-4de8-a568-d3c04e94eab2 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:33.088174 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c094c6-4f81-4de8-a568-d3c04e94eab2 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:33.143970 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b95049-e231-4b28-b48e-9aba2bc9f0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:33.144601 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b95049-e231-4b28-b48e-9aba2bc9f0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:33.144880 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b95049-e231-4b28-b48e-9aba2bc9f0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:33.145151 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b95049-e231-4b28-b48e-9aba2bc9f0db admin admin] PATH_INFO: `/v3/users/204fd056c9e54678adeb14f85a56f005` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:33.146257 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b95049-e231-4b28-b48e-9aba2bc9f0db admin admin] RBAC: Authorizing `identity:delete_user(user_id=204fd056c9e54678adeb14f85a56f005)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:33.148206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b95049-e231-4b28-b48e-9aba2bc9f0db admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:33.189393 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:33.199335 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d0b95049-e231-4b28-b48e-9aba2bc9f0db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '204fd056c9e54678adeb14f85a56f005'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:33.207441 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d0b95049-e231-4b28-b48e-9aba2bc9f0db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '204fd056c9e54678adeb14f85a56f005'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:33.220322 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d0b95049-e231-4b28-b48e-9aba2bc9f0db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '204fd056c9e54678adeb14f85a56f005'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:33.220958 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d0b95049-e231-4b28-b48e-9aba2bc9f0db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '204fd056c9e54678adeb14f85a56f005'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:33.221564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d0b95049-e231-4b28-b48e-9aba2bc9f0db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '204fd056c9e54678adeb14f85a56f005'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:33.234762 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1437/2873] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:33:33 2022] DELETE /identity/v3/users/204fd056c9e54678adeb14f85a56f005 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:33.512080 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b6c48-0560-41a0-9532-d287f0da781a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:33.513838 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-400b6c48-0560-41a0-9532-d287f0da781a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:33.532941 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b6c48-0560-41a0-9532-d287f0da781a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:33.594195 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d589ef-d94a-41f7-b5ca-e77c9fbd2e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:33.594866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d589ef-d94a-41f7-b5ca-e77c9fbd2e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:33.595150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d589ef-d94a-41f7-b5ca-e77c9fbd2e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:33.595428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d589ef-d94a-41f7-b5ca-e77c9fbd2e59 admin admin] PATH_INFO: `/v3/projects/acb811220e8b4d788b628e0aea182921` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:33.596548 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d589ef-d94a-41f7-b5ca-e77c9fbd2e59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acb811220e8b4d788b628e0aea182921)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:33.598467 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d589ef-d94a-41f7-b5ca-e77c9fbd2e59 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:33.645762 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c3d589ef-d94a-41f7-b5ca-e77c9fbd2e59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acb811220e8b4d788b628e0aea182921', 'request_id': 'req-c3d589ef-d94a-41f7-b5ca-e77c9fbd2e59'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:33.652467 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c3d589ef-d94a-41f7-b5ca-e77c9fbd2e59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acb811220e8b4d788b628e0aea182921', 'request_id': 'req-c3d589ef-d94a-41f7-b5ca-e77c9fbd2e59'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:33.657970 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c3d589ef-d94a-41f7-b5ca-e77c9fbd2e59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acb811220e8b4d788b628e0aea182921', 'request_id': 'req-c3d589ef-d94a-41f7-b5ca-e77c9fbd2e59'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:33.671840 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c3d589ef-d94a-41f7-b5ca-e77c9fbd2e59 admin admin] The token cache is being invalidate because project acb811220e8b4d788b628e0aea182921 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:33:33.672129 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c3d589ef-d94a-41f7-b5ca-e77c9fbd2e59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:33.672753 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c3d589ef-d94a-41f7-b5ca-e77c9fbd2e59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:33.674233 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1437/2874] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:33:33 2022] DELETE /identity/v3/projects/acb811220e8b4d788b628e0aea182921 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:33.680123 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13413e1-24a3-44a3-bb3f-46139a756d16 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:33.681324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13413e1-24a3-44a3-bb3f-46139a756d16 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:33.681324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13413e1-24a3-44a3-bb3f-46139a756d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:33.701377 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e13413e1-24a3-44a3-bb3f-46139a756d16 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:33.702774 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e13413e1-24a3-44a3-bb3f-46139a756d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:33.764700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1438/2875] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:33.773132 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2340cbb-882f-4241-9cb4-96439b145c2e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:33.773417 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2340cbb-882f-4241-9cb4-96439b145c2e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:33.773694 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2340cbb-882f-4241-9cb4-96439b145c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:33.795185 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c2340cbb-882f-4241-9cb4-96439b145c2e None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:33.796397 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c2340cbb-882f-4241-9cb4-96439b145c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:33.804421 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1438/2876] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:33.809423 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13413e1-24a3-44a3-bb3f-46139a756d16 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:33.819374 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13413e1-24a3-44a3-bb3f-46139a756d16 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:33.824504 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5a144-c9aa-4faa-aa81-af9fd2de40e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:33.825108 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5a144-c9aa-4faa-aa81-af9fd2de40e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:33.825391 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5a144-c9aa-4faa-aa81-af9fd2de40e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:33.825669 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5a144-c9aa-4faa-aa81-af9fd2de40e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:33.826312 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5a144-c9aa-4faa-aa81-af9fd2de40e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:33:33.826613 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5a144-c9aa-4faa-aa81-af9fd2de40e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:33.828234 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5a144-c9aa-4faa-aa81-af9fd2de40e1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:33.840581 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1439/2877] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:33:33.845218 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2340cbb-882f-4241-9cb4-96439b145c2e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:33.855767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2340cbb-882f-4241-9cb4-96439b145c2e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:33.860634 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee3d3f-3aa6-4055-ad4c-a188bfdad8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:33.861233 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee3d3f-3aa6-4055-ad4c-a188bfdad8e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:33.861516 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee3d3f-3aa6-4055-ad4c-a188bfdad8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:33.861790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee3d3f-3aa6-4055-ad4c-a188bfdad8e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:33.862469 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee3d3f-3aa6-4055-ad4c-a188bfdad8e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:33.864468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee3d3f-3aa6-4055-ad4c-a188bfdad8e8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:33.893172 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1439/2878] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:33:33 2022] POST /identity/v3/projects => generated 359 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:33:33.896939 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5a144-c9aa-4faa-aa81-af9fd2de40e1 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:33.908282 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5a144-c9aa-4faa-aa81-af9fd2de40e1 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:33.965676 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826825b-c605-4324-b46d-0930ffba90a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:33.966275 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826825b-c605-4324-b46d-0930ffba90a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:33.966550 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826825b-c605-4324-b46d-0930ffba90a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:33.966849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826825b-c605-4324-b46d-0930ffba90a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:33.967506 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826825b-c605-4324-b46d-0930ffba90a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:33.969390 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826825b-c605-4324-b46d-0930ffba90a5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:33.991289 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1440/2879] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:33:33 2022] POST /identity/v3/users => generated 280 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:33:34.006174 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee3d3f-3aa6-4055-ad4c-a188bfdad8e8 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:34.018373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee3d3f-3aa6-4055-ad4c-a188bfdad8e8 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:34.023947 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4802be8-db33-433d-8e72-656aeb00416c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:34.024806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4802be8-db33-433d-8e72-656aeb00416c admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:34.025324 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4802be8-db33-433d-8e72-656aeb00416c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:34.025647 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4802be8-db33-433d-8e72-656aeb00416c admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:34.026262 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4802be8-db33-433d-8e72-656aeb00416c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:34.027899 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4802be8-db33-433d-8e72-656aeb00416c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:34.038428 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1440/2880] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:33:34.042617 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826825b-c605-4324-b46d-0930ffba90a5 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:34.052891 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826825b-c605-4324-b46d-0930ffba90a5 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:34.058493 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b4b05-46b1-4b08-a8ad-ec5241ab68fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:34.059119 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b4b05-46b1-4b08-a8ad-ec5241ab68fb admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:34.059399 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b4b05-46b1-4b08-a8ad-ec5241ab68fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:34.059673 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b4b05-46b1-4b08-a8ad-ec5241ab68fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:34.060311 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b4b05-46b1-4b08-a8ad-ec5241ab68fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:34.061887 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b4b05-46b1-4b08-a8ad-ec5241ab68fb admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:34.073297 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1441/2881] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:33:34.077532 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4802be8-db33-433d-8e72-656aeb00416c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:34.088405 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4802be8-db33-433d-8e72-656aeb00416c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:34.093077 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08989dc7-c9f9-4b51-b125-ba81c1f7a33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:34.093720 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08989dc7-c9f9-4b51-b125-ba81c1f7a33e admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:34.093998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08989dc7-c9f9-4b51-b125-ba81c1f7a33e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:34.094273 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08989dc7-c9f9-4b51-b125-ba81c1f7a33e admin admin] PATH_INFO: `/v3/projects/9c5324f451fb4114b493a52dbac40464/users/e4150799bbff44f785f5c8e35347b290/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:34.109594 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08989dc7-c9f9-4b51-b125-ba81c1f7a33e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c5324f451fb4114b493a52dbac40464, user_id=e4150799bbff44f785f5c8e35347b290, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:34.112281 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08989dc7-c9f9-4b51-b125-ba81c1f7a33e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:34.125987 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1441/2882] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:33:34 2022] PUT /identity/v3/projects/9c5324f451fb4114b493a52dbac40464/users/e4150799bbff44f785f5c8e35347b290/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:34.138370 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6aac0-8a80-4739-a27a-b6b1fde8ecc5 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:34.138654 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6aac0-8a80-4739-a27a-b6b1fde8ecc5 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:34.138959 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6aac0-8a80-4739-a27a-b6b1fde8ecc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:34.167904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-6cf6aac0-8a80-4739-a27a-b6b1fde8ecc5 None None] MFA Rules not processed for user `e4150799bbff44f785f5c8e35347b290`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:34.169122 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6cf6aac0-8a80-4739-a27a-b6b1fde8ecc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:34.177324 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:34.236314 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1442/2883] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:34.271877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08989dc7-c9f9-4b51-b125-ba81c1f7a33e admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:34.273507 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-08989dc7-c9f9-4b51-b125-ba81c1f7a33e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:34.293815 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:34.336009 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08989dc7-c9f9-4b51-b125-ba81c1f7a33e admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:34.339928 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c287e6-601d-49ab-8062-d068879b9daf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:34.340530 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c287e6-601d-49ab-8062-d068879b9daf service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:34.340812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c287e6-601d-49ab-8062-d068879b9daf service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:34.341086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c287e6-601d-49ab-8062-d068879b9daf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:34.349818 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c287e6-601d-49ab-8062-d068879b9daf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:34.351291 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c287e6-601d-49ab-8062-d068879b9daf service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:34.360783 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1442/2884] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:34 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:34.376741 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6aac0-8a80-4739-a27a-b6b1fde8ecc5 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:34.440886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6aac0-8a80-4739-a27a-b6b1fde8ecc5 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:34.445406 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec213f-2467-4433-9eaf-bb55d2d49d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:34.446029 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec213f-2467-4433-9eaf-bb55d2d49d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:34.446425 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec213f-2467-4433-9eaf-bb55d2d49d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:34.446758 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec213f-2467-4433-9eaf-bb55d2d49d1d admin admin] PATH_INFO: `/v3/users/e4150799bbff44f785f5c8e35347b290` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:34.447886 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec213f-2467-4433-9eaf-bb55d2d49d1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4150799bbff44f785f5c8e35347b290)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:34.449800 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec213f-2467-4433-9eaf-bb55d2d49d1d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:34.496417 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-37ec213f-2467-4433-9eaf-bb55d2d49d1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4150799bbff44f785f5c8e35347b290'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:34.503356 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-37ec213f-2467-4433-9eaf-bb55d2d49d1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4150799bbff44f785f5c8e35347b290'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:34.515434 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-37ec213f-2467-4433-9eaf-bb55d2d49d1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4150799bbff44f785f5c8e35347b290'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:34.516125 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-37ec213f-2467-4433-9eaf-bb55d2d49d1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4150799bbff44f785f5c8e35347b290'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:34.516785 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-37ec213f-2467-4433-9eaf-bb55d2d49d1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4150799bbff44f785f5c8e35347b290'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:34.531152 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1443/2885] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:33:34 2022] DELETE /identity/v3/users/e4150799bbff44f785f5c8e35347b290 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:34.536395 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b9a12-1d06-4721-b9b1-a9a118d4fff2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:34.536961 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b9a12-1d06-4721-b9b1-a9a118d4fff2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:34.536961 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b9a12-1d06-4721-b9b1-a9a118d4fff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:34.557343 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6e8b9a12-1d06-4721-b9b1-a9a118d4fff2 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:34.558557 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6e8b9a12-1d06-4721-b9b1-a9a118d4fff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:34.617225 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1443/2886] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:34.629628 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec213f-2467-4433-9eaf-bb55d2d49d1d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:34.631206 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-37ec213f-2467-4433-9eaf-bb55d2d49d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:34.663055 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:34.690849 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec213f-2467-4433-9eaf-bb55d2d49d1d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:34.694593 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76847b5-2f8a-4545-b393-c1ba737807ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:34.695225 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76847b5-2f8a-4545-b393-c1ba737807ae service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:34.695505 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76847b5-2f8a-4545-b393-c1ba737807ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:34.695793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76847b5-2f8a-4545-b393-c1ba737807ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:34.704870 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76847b5-2f8a-4545-b393-c1ba737807ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:34.706308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76847b5-2f8a-4545-b393-c1ba737807ae service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:34.716519 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1444/2887] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:34.985166 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b9a12-1d06-4721-b9b1-a9a118d4fff2 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:34.986651 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6e8b9a12-1d06-4721-b9b1-a9a118d4fff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:34.998912 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b9a12-1d06-4721-b9b1-a9a118d4fff2 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:35.003666 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb54ca5-a031-4a40-884e-c75166f30b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:35.004308 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb54ca5-a031-4a40-884e-c75166f30b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:35.004596 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb54ca5-a031-4a40-884e-c75166f30b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:35.004870 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb54ca5-a031-4a40-884e-c75166f30b18 admin admin] PATH_INFO: `/v3/projects/9c5324f451fb4114b493a52dbac40464` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:35.006197 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb54ca5-a031-4a40-884e-c75166f30b18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c5324f451fb4114b493a52dbac40464)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:35.008473 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb54ca5-a031-4a40-884e-c75166f30b18 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:35.057313 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7eb54ca5-a031-4a40-884e-c75166f30b18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c5324f451fb4114b493a52dbac40464', 'request_id': 'req-7eb54ca5-a031-4a40-884e-c75166f30b18'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:35.063892 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7eb54ca5-a031-4a40-884e-c75166f30b18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c5324f451fb4114b493a52dbac40464', 'request_id': 'req-7eb54ca5-a031-4a40-884e-c75166f30b18'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:35.069866 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7eb54ca5-a031-4a40-884e-c75166f30b18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c5324f451fb4114b493a52dbac40464', 'request_id': 'req-7eb54ca5-a031-4a40-884e-c75166f30b18'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:35.083112 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7eb54ca5-a031-4a40-884e-c75166f30b18 admin admin] The token cache is being invalidate because project 9c5324f451fb4114b493a52dbac40464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:33:35.083426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7eb54ca5-a031-4a40-884e-c75166f30b18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:35.084048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7eb54ca5-a031-4a40-884e-c75166f30b18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:35.085861 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1444/2888] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:33:34 2022] DELETE /identity/v3/projects/9c5324f451fb4114b493a52dbac40464 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:35.091301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03faa1fa-33ea-4749-987a-74538d2765db None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:35.091588 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03faa1fa-33ea-4749-987a-74538d2765db None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:35.091860 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03faa1fa-33ea-4749-987a-74538d2765db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:35.111343 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-03faa1fa-33ea-4749-987a-74538d2765db None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:35.112541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-03faa1fa-33ea-4749-987a-74538d2765db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:35.175793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1445/2889] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:35.187530 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e89b6c-b4b4-432e-8902-d8c30b54247f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:35.188099 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e89b6c-b4b4-432e-8902-d8c30b54247f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:35.188382 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e89b6c-b4b4-432e-8902-d8c30b54247f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:35.209049 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-63e89b6c-b4b4-432e-8902-d8c30b54247f None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:35.210539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-63e89b6c-b4b4-432e-8902-d8c30b54247f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:35.219528 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1445/2890] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:35.223855 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03faa1fa-33ea-4749-987a-74538d2765db None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:35.233822 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03faa1fa-33ea-4749-987a-74538d2765db None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:35.238584 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc259f-488c-443e-b916-ccb6bea9031d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:35.239218 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc259f-488c-443e-b916-ccb6bea9031d admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:35.239496 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc259f-488c-443e-b916-ccb6bea9031d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:35.239767 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc259f-488c-443e-b916-ccb6bea9031d admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:35.240413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc259f-488c-443e-b916-ccb6bea9031d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:33:35.240717 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc259f-488c-443e-b916-ccb6bea9031d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:35.242309 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc259f-488c-443e-b916-ccb6bea9031d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:35.254541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1446/2891] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:33:35.260537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e89b6c-b4b4-432e-8902-d8c30b54247f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:35.276931 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e89b6c-b4b4-432e-8902-d8c30b54247f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:35.282699 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50b41c-15bd-4d49-b66f-d80ff1821e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:35.283403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50b41c-15bd-4d49-b66f-d80ff1821e14 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:35.283725 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50b41c-15bd-4d49-b66f-d80ff1821e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:35.284039 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50b41c-15bd-4d49-b66f-d80ff1821e14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:35.284734 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50b41c-15bd-4d49-b66f-d80ff1821e14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:35.286884 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50b41c-15bd-4d49-b66f-d80ff1821e14 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:35.316004 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1446/2892] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:33:35 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:33:35.320147 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc259f-488c-443e-b916-ccb6bea9031d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:35.331021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc259f-488c-443e-b916-ccb6bea9031d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:35.388271 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad88d7-b3fe-4622-b8d8-ba3ed26e98ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:35.388889 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad88d7-b3fe-4622-b8d8-ba3ed26e98ef admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:35.389172 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad88d7-b3fe-4622-b8d8-ba3ed26e98ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:35.389453 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad88d7-b3fe-4622-b8d8-ba3ed26e98ef admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:35.390082 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad88d7-b3fe-4622-b8d8-ba3ed26e98ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:35.392036 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad88d7-b3fe-4622-b8d8-ba3ed26e98ef admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:35.413988 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1447/2893] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:33:35 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:33:35.418741 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50b41c-15bd-4d49-b66f-d80ff1821e14 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:35.428958 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50b41c-15bd-4d49-b66f-d80ff1821e14 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:35.434326 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb09e66-99d2-4c2c-adfb-6291818e539f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:35.434950 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb09e66-99d2-4c2c-adfb-6291818e539f admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:35.435232 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb09e66-99d2-4c2c-adfb-6291818e539f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:35.435504 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb09e66-99d2-4c2c-adfb-6291818e539f admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:35.436142 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb09e66-99d2-4c2c-adfb-6291818e539f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:35.437732 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb09e66-99d2-4c2c-adfb-6291818e539f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:35.447791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1447/2894] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:33:35.451850 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad88d7-b3fe-4622-b8d8-ba3ed26e98ef admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:35.462841 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad88d7-b3fe-4622-b8d8-ba3ed26e98ef admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:35.467732 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a16f28-50a1-40ea-93c9-f72f2d2add6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:35.468345 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a16f28-50a1-40ea-93c9-f72f2d2add6b admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:35.468651 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a16f28-50a1-40ea-93c9-f72f2d2add6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:35.468957 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a16f28-50a1-40ea-93c9-f72f2d2add6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:35.469678 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a16f28-50a1-40ea-93c9-f72f2d2add6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:35.471502 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a16f28-50a1-40ea-93c9-f72f2d2add6b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:35.484358 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1448/2895] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:33:35.489399 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb09e66-99d2-4c2c-adfb-6291818e539f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:35.494744 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:35.501283 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb09e66-99d2-4c2c-adfb-6291818e539f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:35.507634 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef32c6-966a-4302-8986-f49bd722a452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:35.508381 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef32c6-966a-4302-8986-f49bd722a452 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:35.508671 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef32c6-966a-4302-8986-f49bd722a452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:35.508977 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef32c6-966a-4302-8986-f49bd722a452 admin admin] PATH_INFO: `/v3/projects/03894a1e23744d5e97a65a6704fa86bd/users/c719d3c801554050beb51c18064f1a04/roles/e89162686187404da84651e0a3632b7b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:35.524214 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef32c6-966a-4302-8986-f49bd722a452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03894a1e23744d5e97a65a6704fa86bd, user_id=c719d3c801554050beb51c18064f1a04, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:35.527564 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef32c6-966a-4302-8986-f49bd722a452 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:35.541951 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1448/2896] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:33:35 2022] PUT /identity/v3/projects/03894a1e23744d5e97a65a6704fa86bd/users/c719d3c801554050beb51c18064f1a04/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:35.559205 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f03abe-074d-440e-8dcf-115df15128ea None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:35.559516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f03abe-074d-440e-8dcf-115df15128ea None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:35.559793 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f03abe-074d-440e-8dcf-115df15128ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:35.588622 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f2f03abe-074d-440e-8dcf-115df15128ea None None] MFA Rules not processed for user `c719d3c801554050beb51c18064f1a04`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:35.589819 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f2f03abe-074d-440e-8dcf-115df15128ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:35.616555 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:35.656657 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1449/2897] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:33:35.661251 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef32c6-966a-4302-8986-f49bd722a452 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:35.725483 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef32c6-966a-4302-8986-f49bd722a452 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:35.730286 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ed4eb-5a19-46f2-8bb9-f1b94022a727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:35.730918 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ed4eb-5a19-46f2-8bb9-f1b94022a727 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:35.731198 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ed4eb-5a19-46f2-8bb9-f1b94022a727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:35.731468 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ed4eb-5a19-46f2-8bb9-f1b94022a727 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:35.732096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ed4eb-5a19-46f2-8bb9-f1b94022a727 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:35.733995 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ed4eb-5a19-46f2-8bb9-f1b94022a727 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:35.761373 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1449/2898] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:33:35 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:33:35.765520 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f03abe-074d-440e-8dcf-115df15128ea None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:35.776454 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f03abe-074d-440e-8dcf-115df15128ea None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:35.834234 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13f206-092a-4872-a3fa-c03613c7a1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:35.834900 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13f206-092a-4872-a3fa-c03613c7a1d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:35.835433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13f206-092a-4872-a3fa-c03613c7a1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:35.835713 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13f206-092a-4872-a3fa-c03613c7a1d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:35.836350 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13f206-092a-4872-a3fa-c03613c7a1d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:35.838400 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13f206-092a-4872-a3fa-c03613c7a1d4 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:35.860702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1450/2899] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:33:35 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:33:35.864707 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ed4eb-5a19-46f2-8bb9-f1b94022a727 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:35.876391 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ed4eb-5a19-46f2-8bb9-f1b94022a727 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:35.881008 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f96418-46a0-41d5-bb1b-6f2a533973da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:35.881598 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f96418-46a0-41d5-bb1b-6f2a533973da admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:35.881877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f96418-46a0-41d5-bb1b-6f2a533973da admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:35.882149 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f96418-46a0-41d5-bb1b-6f2a533973da admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:35.882850 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f96418-46a0-41d5-bb1b-6f2a533973da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:35.885166 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f96418-46a0-41d5-bb1b-6f2a533973da admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:35.896294 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1450/2900] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:33:35.900648 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13f206-092a-4872-a3fa-c03613c7a1d4 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:35.910581 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13f206-092a-4872-a3fa-c03613c7a1d4 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:35.916149 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3dc1c4-7ae9-4735-802a-6e18519f751d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:35.917812 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3dc1c4-7ae9-4735-802a-6e18519f751d admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:35.917812 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3dc1c4-7ae9-4735-802a-6e18519f751d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:35.917812 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3dc1c4-7ae9-4735-802a-6e18519f751d admin admin] PATH_INFO: `/v3/projects/01c04cb93b644780ab7ec310bfe2a068/users/cdea7f6d04f7436a8a344ed072cca8ce/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:35.932702 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3dc1c4-7ae9-4735-802a-6e18519f751d admin admin] RBAC: Authorizing `identity:create_grant(project_id=01c04cb93b644780ab7ec310bfe2a068, user_id=cdea7f6d04f7436a8a344ed072cca8ce, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:35.935381 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3dc1c4-7ae9-4735-802a-6e18519f751d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:35.949277 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1451/2901] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:33:35 2022] PUT /identity/v3/projects/01c04cb93b644780ab7ec310bfe2a068/users/cdea7f6d04f7436a8a344ed072cca8ce/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:33:35.953232 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f96418-46a0-41d5-bb1b-6f2a533973da admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:35.963861 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f96418-46a0-41d5-bb1b-6f2a533973da admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:36.021877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f843b12-8048-4f4b-a047-d632072fe727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:36.022486 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f843b12-8048-4f4b-a047-d632072fe727 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:36.022796 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f843b12-8048-4f4b-a047-d632072fe727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:36.023071 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f843b12-8048-4f4b-a047-d632072fe727 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:36.023709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f843b12-8048-4f4b-a047-d632072fe727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:36.025862 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f843b12-8048-4f4b-a047-d632072fe727 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:36.035943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1451/2902] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:33:36.041359 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3dc1c4-7ae9-4735-802a-6e18519f751d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:36.051813 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3dc1c4-7ae9-4735-802a-6e18519f751d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:36.058307 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578eb4c4-30ea-445a-ba65-7154f5c2d964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:36.059016 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578eb4c4-30ea-445a-ba65-7154f5c2d964 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:36.059339 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578eb4c4-30ea-445a-ba65-7154f5c2d964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:36.059637 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578eb4c4-30ea-445a-ba65-7154f5c2d964 admin admin] PATH_INFO: `/v3/projects/01c04cb93b644780ab7ec310bfe2a068/users/cdea7f6d04f7436a8a344ed072cca8ce/roles/ba05677650e046a48d5f19440d48fbf2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:36.061541 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578eb4c4-30ea-445a-ba65-7154f5c2d964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01c04cb93b644780ab7ec310bfe2a068, user_id=cdea7f6d04f7436a8a344ed072cca8ce, role_id=ba05677650e046a48d5f19440d48fbf2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:36.064386 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578eb4c4-30ea-445a-ba65-7154f5c2d964 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:36.079005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1452/2903] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:33:36 2022] PUT /identity/v3/projects/01c04cb93b644780ab7ec310bfe2a068/users/cdea7f6d04f7436a8a344ed072cca8ce/roles/ba05677650e046a48d5f19440d48fbf2 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:33:36.089626 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5be5c5-5336-49c8-8d1d-376a476078f1 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:36.089923 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5be5c5-5336-49c8-8d1d-376a476078f1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:36.090201 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5be5c5-5336-49c8-8d1d-376a476078f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:36.121320 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-9a5be5c5-5336-49c8-8d1d-376a476078f1 None None] MFA Rules not processed for user `cdea7f6d04f7436a8a344ed072cca8ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:36.122584 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9a5be5c5-5336-49c8-8d1d-376a476078f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:36.166813 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:36.199790 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1452/2904] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:36.211064 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578eb4c4-30ea-445a-ba65-7154f5c2d964 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:36.214168 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-578eb4c4-30ea-445a-ba65-7154f5c2d964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:36.278378 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578eb4c4-30ea-445a-ba65-7154f5c2d964 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:36.282130 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43225d-b94f-49df-8002-1876211807b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:36.282763 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43225d-b94f-49df-8002-1876211807b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:36.283080 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43225d-b94f-49df-8002-1876211807b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:36.283352 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43225d-b94f-49df-8002-1876211807b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:36.317299 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:36.344962 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e43225d-b94f-49df-8002-1876211807b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:36.346465 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e43225d-b94f-49df-8002-1876211807b0 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:36.356988 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1453/2905] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:38.932635 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5be5c5-5336-49c8-8d1d-376a476078f1 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:39.019398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5be5c5-5336-49c8-8d1d-376a476078f1 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:39.024302 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344495d-ba62-44fb-af92-5d32024e5a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:39.024959 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344495d-ba62-44fb-af92-5d32024e5a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:39.025233 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344495d-ba62-44fb-af92-5d32024e5a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:39.025676 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344495d-ba62-44fb-af92-5d32024e5a7e admin admin] PATH_INFO: `/v3/users/c719d3c801554050beb51c18064f1a04` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:39.027757 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344495d-ba62-44fb-af92-5d32024e5a7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c719d3c801554050beb51c18064f1a04)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:39.033491 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344495d-ba62-44fb-af92-5d32024e5a7e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:39.085652 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7344495d-ba62-44fb-af92-5d32024e5a7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c719d3c801554050beb51c18064f1a04'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:39.100831 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7344495d-ba62-44fb-af92-5d32024e5a7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c719d3c801554050beb51c18064f1a04'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:39.113145 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7344495d-ba62-44fb-af92-5d32024e5a7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c719d3c801554050beb51c18064f1a04'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:39.113892 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7344495d-ba62-44fb-af92-5d32024e5a7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c719d3c801554050beb51c18064f1a04'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:39.121612 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7344495d-ba62-44fb-af92-5d32024e5a7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c719d3c801554050beb51c18064f1a04'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:39.127867 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1453/2906] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:33:38 2022] DELETE /identity/v3/users/c719d3c801554050beb51c18064f1a04 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:33:39.132407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbac105-a385-402d-b9bc-9cc51ce68c27 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:39.132904 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbac105-a385-402d-b9bc-9cc51ce68c27 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:39.133011 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbac105-a385-402d-b9bc-9cc51ce68c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:39.167441 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9dbac105-a385-402d-b9bc-9cc51ce68c27 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:39.168548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9dbac105-a385-402d-b9bc-9cc51ce68c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:39.229321 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1454/2907] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:39.239550 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344495d-ba62-44fb-af92-5d32024e5a7e admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:39.240943 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7344495d-ba62-44fb-af92-5d32024e5a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:39.281258 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:39.323225 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344495d-ba62-44fb-af92-5d32024e5a7e admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:39.327012 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85995fe9-c271-42a5-b5c1-1341d66491c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:39.327609 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85995fe9-c271-42a5-b5c1-1341d66491c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:39.327889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85995fe9-c271-42a5-b5c1-1341d66491c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:39.328163 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85995fe9-c271-42a5-b5c1-1341d66491c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:39.337766 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85995fe9-c271-42a5-b5c1-1341d66491c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:39.339729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85995fe9-c271-42a5-b5c1-1341d66491c0 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:39.349306 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1454/2908] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:39.630569 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbac105-a385-402d-b9bc-9cc51ce68c27 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:39.632072 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9dbac105-a385-402d-b9bc-9cc51ce68c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:39.645350 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbac105-a385-402d-b9bc-9cc51ce68c27 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:39.650203 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bd2bb-3b3f-42b0-9638-f282cab403cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:39.652025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bd2bb-3b3f-42b0-9638-f282cab403cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:39.652304 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bd2bb-3b3f-42b0-9638-f282cab403cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:39.652578 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bd2bb-3b3f-42b0-9638-f282cab403cd admin admin] PATH_INFO: `/v3/projects/03894a1e23744d5e97a65a6704fa86bd` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:39.653726 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bd2bb-3b3f-42b0-9638-f282cab403cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=03894a1e23744d5e97a65a6704fa86bd)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:39.655755 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bd2bb-3b3f-42b0-9638-f282cab403cd admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:39.703176 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-856bd2bb-3b3f-42b0-9638-f282cab403cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03894a1e23744d5e97a65a6704fa86bd', 'request_id': 'req-856bd2bb-3b3f-42b0-9638-f282cab403cd'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:39.710501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-856bd2bb-3b3f-42b0-9638-f282cab403cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03894a1e23744d5e97a65a6704fa86bd', 'request_id': 'req-856bd2bb-3b3f-42b0-9638-f282cab403cd'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:39.717954 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-856bd2bb-3b3f-42b0-9638-f282cab403cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03894a1e23744d5e97a65a6704fa86bd', 'request_id': 'req-856bd2bb-3b3f-42b0-9638-f282cab403cd'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:39.729484 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-856bd2bb-3b3f-42b0-9638-f282cab403cd admin admin] The token cache is being invalidate because project 03894a1e23744d5e97a65a6704fa86bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:33:39.729780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-856bd2bb-3b3f-42b0-9638-f282cab403cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:39.730426 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-856bd2bb-3b3f-42b0-9638-f282cab403cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:39.731882 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1455/2909] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:33:39 2022] DELETE /identity/v3/projects/03894a1e23744d5e97a65a6704fa86bd => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:39.735814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85995fe9-c271-42a5-b5c1-1341d66491c0 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:39.746340 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85995fe9-c271-42a5-b5c1-1341d66491c0 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:39.803816 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da663999-4533-4cb6-8e46-f1c89552f2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:39.804440 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da663999-4533-4cb6-8e46-f1c89552f2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:39.804718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da663999-4533-4cb6-8e46-f1c89552f2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:39.805020 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da663999-4533-4cb6-8e46-f1c89552f2cf admin admin] PATH_INFO: `/v3/users/cdea7f6d04f7436a8a344ed072cca8ce` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:39.806129 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da663999-4533-4cb6-8e46-f1c89552f2cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdea7f6d04f7436a8a344ed072cca8ce)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:39.808543 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da663999-4533-4cb6-8e46-f1c89552f2cf admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:39.862332 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-da663999-4533-4cb6-8e46-f1c89552f2cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdea7f6d04f7436a8a344ed072cca8ce'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:39.869364 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-da663999-4533-4cb6-8e46-f1c89552f2cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdea7f6d04f7436a8a344ed072cca8ce'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:39.882096 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-da663999-4533-4cb6-8e46-f1c89552f2cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdea7f6d04f7436a8a344ed072cca8ce'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:39.882888 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-da663999-4533-4cb6-8e46-f1c89552f2cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdea7f6d04f7436a8a344ed072cca8ce'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:39.891265 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-da663999-4533-4cb6-8e46-f1c89552f2cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdea7f6d04f7436a8a344ed072cca8ce'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:39.897843 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1455/2910] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:33:39 2022] DELETE /identity/v3/users/cdea7f6d04f7436a8a344ed072cca8ce => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:40.174468 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bd2bb-3b3f-42b0-9638-f282cab403cd admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:40.176012 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-856bd2bb-3b3f-42b0-9638-f282cab403cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:40.190164 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bd2bb-3b3f-42b0-9638-f282cab403cd admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:40.237181 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:40.251561 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ddc2e-b1ad-4ee0-8cdb-ecdffec79144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:40.252129 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ddc2e-b1ad-4ee0-8cdb-ecdffec79144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:40.252407 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ddc2e-b1ad-4ee0-8cdb-ecdffec79144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:40.252677 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ddc2e-b1ad-4ee0-8cdb-ecdffec79144 admin admin] PATH_INFO: `/v3/projects/01c04cb93b644780ab7ec310bfe2a068` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:40.253840 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ddc2e-b1ad-4ee0-8cdb-ecdffec79144 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01c04cb93b644780ab7ec310bfe2a068)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:40.255795 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ddc2e-b1ad-4ee0-8cdb-ecdffec79144 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:40.304028 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b65ddc2e-b1ad-4ee0-8cdb-ecdffec79144 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01c04cb93b644780ab7ec310bfe2a068', 'request_id': 'req-b65ddc2e-b1ad-4ee0-8cdb-ecdffec79144'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:40.311689 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b65ddc2e-b1ad-4ee0-8cdb-ecdffec79144 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01c04cb93b644780ab7ec310bfe2a068', 'request_id': 'req-b65ddc2e-b1ad-4ee0-8cdb-ecdffec79144'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:40.318257 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b65ddc2e-b1ad-4ee0-8cdb-ecdffec79144 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01c04cb93b644780ab7ec310bfe2a068', 'request_id': 'req-b65ddc2e-b1ad-4ee0-8cdb-ecdffec79144'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:40.329723 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b65ddc2e-b1ad-4ee0-8cdb-ecdffec79144 admin admin] The token cache is being invalidate because project 01c04cb93b644780ab7ec310bfe2a068 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:33:40.330009 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b65ddc2e-b1ad-4ee0-8cdb-ecdffec79144 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:40.330724 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b65ddc2e-b1ad-4ee0-8cdb-ecdffec79144 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:40.332200 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1456/2911] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:33:40 2022] DELETE /identity/v3/projects/01c04cb93b644780ab7ec310bfe2a068 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:40.337293 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3539d72-68ff-4413-8f4b-4b57ac04fa42 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:40.337805 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3539d72-68ff-4413-8f4b-4b57ac04fa42 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:40.337920 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3539d72-68ff-4413-8f4b-4b57ac04fa42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:40.359479 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a3539d72-68ff-4413-8f4b-4b57ac04fa42 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:40.360974 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a3539d72-68ff-4413-8f4b-4b57ac04fa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:40.393801 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:40.423810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1456/2912] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:40.431501 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7c342-f9ae-4849-ae16-861a69b9e148 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:40.432421 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7c342-f9ae-4849-ae16-861a69b9e148 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:40.432703 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7c342-f9ae-4849-ae16-861a69b9e148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:40.452279 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f8f7c342-f9ae-4849-ae16-861a69b9e148 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:40.453485 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f8f7c342-f9ae-4849-ae16-861a69b9e148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:40.461610 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1457/2913] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:40.466207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3539d72-68ff-4413-8f4b-4b57ac04fa42 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:40.476056 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3539d72-68ff-4413-8f4b-4b57ac04fa42 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:40.481247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8bb59-071d-4124-b606-b6184fbff6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:40.481836 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8bb59-071d-4124-b606-b6184fbff6ec admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:40.482116 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8bb59-071d-4124-b606-b6184fbff6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:40.482392 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8bb59-071d-4124-b606-b6184fbff6ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:40.483083 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8bb59-071d-4124-b606-b6184fbff6ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:33:40.483387 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8bb59-071d-4124-b606-b6184fbff6ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:40.484972 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8bb59-071d-4124-b606-b6184fbff6ec admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:40.497073 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1457/2914] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:33:40.501433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7c342-f9ae-4849-ae16-861a69b9e148 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:40.510889 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7c342-f9ae-4849-ae16-861a69b9e148 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:40.515516 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f585bb-a833-42dd-9511-957d106a8257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:40.516111 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f585bb-a833-42dd-9511-957d106a8257 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:40.516392 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f585bb-a833-42dd-9511-957d106a8257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:40.516665 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f585bb-a833-42dd-9511-957d106a8257 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:40.517302 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f585bb-a833-42dd-9511-957d106a8257 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:40.519235 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f585bb-a833-42dd-9511-957d106a8257 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:40.546820 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1458/2915] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:33:40 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 16:33:40.551176 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8bb59-071d-4124-b606-b6184fbff6ec admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:40.562260 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8bb59-071d-4124-b606-b6184fbff6ec admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:40.621349 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347fc3e-4f08-4960-ab22-6ae792895c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:40.621961 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347fc3e-4f08-4960-ab22-6ae792895c92 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:40.622239 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347fc3e-4f08-4960-ab22-6ae792895c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:40.622513 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347fc3e-4f08-4960-ab22-6ae792895c92 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:40.623207 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347fc3e-4f08-4960-ab22-6ae792895c92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:40.625172 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347fc3e-4f08-4960-ab22-6ae792895c92 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:40.646171 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1458/2916] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:33:40 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:33:40.650268 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f585bb-a833-42dd-9511-957d106a8257 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:40.661072 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f585bb-a833-42dd-9511-957d106a8257 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:40.665552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756857a-51cb-48c6-868a-3bf458fba57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:40.666720 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756857a-51cb-48c6-868a-3bf458fba57f admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:40.667038 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756857a-51cb-48c6-868a-3bf458fba57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:40.667344 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756857a-51cb-48c6-868a-3bf458fba57f admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:40.667939 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c756857a-51cb-48c6-868a-3bf458fba57f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:40.669507 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c756857a-51cb-48c6-868a-3bf458fba57f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:40.679535 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1459/2917] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:33:40.684048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347fc3e-4f08-4960-ab22-6ae792895c92 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:40.693810 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347fc3e-4f08-4960-ab22-6ae792895c92 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:40.698430 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac10ed3-a0f3-4212-874a-7eaf8bb0f622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:40.699082 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac10ed3-a0f3-4212-874a-7eaf8bb0f622 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:40.699380 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac10ed3-a0f3-4212-874a-7eaf8bb0f622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:40.699698 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac10ed3-a0f3-4212-874a-7eaf8bb0f622 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:40.700398 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac10ed3-a0f3-4212-874a-7eaf8bb0f622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:40.701961 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac10ed3-a0f3-4212-874a-7eaf8bb0f622 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:40.712783 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1459/2918] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:33:40.717711 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756857a-51cb-48c6-868a-3bf458fba57f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:40.727931 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756857a-51cb-48c6-868a-3bf458fba57f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:40.733254 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285aed7-2171-4263-a2c3-f1f15fee4c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:40.733906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285aed7-2171-4263-a2c3-f1f15fee4c01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:40.734185 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285aed7-2171-4263-a2c3-f1f15fee4c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:40.734458 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285aed7-2171-4263-a2c3-f1f15fee4c01 admin admin] PATH_INFO: `/v3/projects/8221960d710a4b319f35da195edeee45/users/e9bf376e527343e58574afc39bc74467/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:40.749375 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285aed7-2171-4263-a2c3-f1f15fee4c01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8221960d710a4b319f35da195edeee45, user_id=e9bf376e527343e58574afc39bc74467, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:40.752066 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285aed7-2171-4263-a2c3-f1f15fee4c01 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:40.764919 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1460/2919] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:33:40 2022] PUT /identity/v3/projects/8221960d710a4b319f35da195edeee45/users/e9bf376e527343e58574afc39bc74467/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:40.777917 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1276c-dc25-4bbd-9e08-7475e56405b4 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:40.778247 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1276c-dc25-4bbd-9e08-7475e56405b4 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:40.778562 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1276c-dc25-4bbd-9e08-7475e56405b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:40.807772 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-9ed1276c-dc25-4bbd-9e08-7475e56405b4 None None] MFA Rules not processed for user `e9bf376e527343e58574afc39bc74467`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:40.809020 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9ed1276c-dc25-4bbd-9e08-7475e56405b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:40.873709 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1460/2920] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:40.884381 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285aed7-2171-4263-a2c3-f1f15fee4c01 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:40.886606 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7285aed7-2171-4263-a2c3-f1f15fee4c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:40.944630 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285aed7-2171-4263-a2c3-f1f15fee4c01 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:40.948453 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55fb83-41d0-4cdb-89e5-c4729b9d4213 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:40.949057 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55fb83-41d0-4cdb-89e5-c4729b9d4213 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:40.949342 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55fb83-41d0-4cdb-89e5-c4729b9d4213 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:40.949616 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55fb83-41d0-4cdb-89e5-c4729b9d4213 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:40.957906 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55fb83-41d0-4cdb-89e5-c4729b9d4213 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:40.959383 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55fb83-41d0-4cdb-89e5-c4729b9d4213 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:40.969461 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1461/2921] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:50.870993 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=82,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:50.965297 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:51.333807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1276c-dc25-4bbd-9e08-7475e56405b4 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:51.397280 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1276c-dc25-4bbd-9e08-7475e56405b4 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:51.402074 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794a340-fc5a-4e5f-8ca4-ab182ee15d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:51.402729 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794a340-fc5a-4e5f-8ca4-ab182ee15d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:51.403014 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794a340-fc5a-4e5f-8ca4-ab182ee15d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:51.403297 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794a340-fc5a-4e5f-8ca4-ab182ee15d80 admin admin] PATH_INFO: `/v3/users/e9bf376e527343e58574afc39bc74467` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:51.404429 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794a340-fc5a-4e5f-8ca4-ab182ee15d80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9bf376e527343e58574afc39bc74467)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:51.406344 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794a340-fc5a-4e5f-8ca4-ab182ee15d80 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:51.455947 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c794a340-fc5a-4e5f-8ca4-ab182ee15d80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9bf376e527343e58574afc39bc74467'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:51.463553 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c794a340-fc5a-4e5f-8ca4-ab182ee15d80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9bf376e527343e58574afc39bc74467'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:51.476003 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c794a340-fc5a-4e5f-8ca4-ab182ee15d80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9bf376e527343e58574afc39bc74467'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:51.476758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c794a340-fc5a-4e5f-8ca4-ab182ee15d80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9bf376e527343e58574afc39bc74467'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:51.484461 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c794a340-fc5a-4e5f-8ca4-ab182ee15d80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9bf376e527343e58574afc39bc74467'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:51.490610 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1461/2922] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:33:51 2022] DELETE /identity/v3/users/e9bf376e527343e58574afc39bc74467 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:51.495058 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7ef59-e11b-413a-a925-8f0a72988d3e None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:51.495864 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7ef59-e11b-413a-a925-8f0a72988d3e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:51.496551 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7ef59-e11b-413a-a925-8f0a72988d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:51.515368 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-faf7ef59-e11b-413a-a925-8f0a72988d3e None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:51.516571 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-faf7ef59-e11b-413a-a925-8f0a72988d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:51.582566 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1462/2923] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:51.592704 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794a340-fc5a-4e5f-8ca4-ab182ee15d80 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:51.594086 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c794a340-fc5a-4e5f-8ca4-ab182ee15d80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:51.653782 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794a340-fc5a-4e5f-8ca4-ab182ee15d80 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:51.658306 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ffe4d1-fda8-4df9-98be-15f0f80da392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:51.658940 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffe4d1-fda8-4df9-98be-15f0f80da392 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:51.659225 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffe4d1-fda8-4df9-98be-15f0f80da392 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:51.659499 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffe4d1-fda8-4df9-98be-15f0f80da392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:51.669043 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ffe4d1-fda8-4df9-98be-15f0f80da392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:51.670484 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ffe4d1-fda8-4df9-98be-15f0f80da392 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:51.680025 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1462/2924] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:33:51.952224 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7ef59-e11b-413a-a925-8f0a72988d3e None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:51.953753 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-faf7ef59-e11b-413a-a925-8f0a72988d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:51.966477 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7ef59-e11b-413a-a925-8f0a72988d3e None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:51.971386 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2227cb-9b49-4ad9-a0fd-213b96fb3dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:51.972025 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2227cb-9b49-4ad9-a0fd-213b96fb3dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:51.972301 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2227cb-9b49-4ad9-a0fd-213b96fb3dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:51.972568 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2227cb-9b49-4ad9-a0fd-213b96fb3dae admin admin] PATH_INFO: `/v3/projects/8221960d710a4b319f35da195edeee45` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:51.973678 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2227cb-9b49-4ad9-a0fd-213b96fb3dae admin admin] RBAC: Authorizing `identity:delete_project(project_id=8221960d710a4b319f35da195edeee45)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:51.975629 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2227cb-9b49-4ad9-a0fd-213b96fb3dae admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:52.023772 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce2227cb-9b49-4ad9-a0fd-213b96fb3dae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8221960d710a4b319f35da195edeee45', 'request_id': 'req-ce2227cb-9b49-4ad9-a0fd-213b96fb3dae'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:52.031155 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce2227cb-9b49-4ad9-a0fd-213b96fb3dae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8221960d710a4b319f35da195edeee45', 'request_id': 'req-ce2227cb-9b49-4ad9-a0fd-213b96fb3dae'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:52.037320 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce2227cb-9b49-4ad9-a0fd-213b96fb3dae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8221960d710a4b319f35da195edeee45', 'request_id': 'req-ce2227cb-9b49-4ad9-a0fd-213b96fb3dae'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:52.048974 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce2227cb-9b49-4ad9-a0fd-213b96fb3dae admin admin] The token cache is being invalidate because project 8221960d710a4b319f35da195edeee45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:33:52.049260 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce2227cb-9b49-4ad9-a0fd-213b96fb3dae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:52.049932 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce2227cb-9b49-4ad9-a0fd-213b96fb3dae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:33:52.051460 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1463/2925] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:33:51 2022] DELETE /identity/v3/projects/8221960d710a4b319f35da195edeee45 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:33:52.056537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6f5f5-05cd-490b-8d0b-f602e2e9b5b3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:52.057042 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6f5f5-05cd-490b-8d0b-f602e2e9b5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:52.057169 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6f5f5-05cd-490b-8d0b-f602e2e9b5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:52.078902 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-01a6f5f5-05cd-490b-8d0b-f602e2e9b5b3 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:52.080198 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-01a6f5f5-05cd-490b-8d0b-f602e2e9b5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:52.098539 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:52.143521 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1463/2926] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:52.151424 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcdc9c-10d2-4e04-af93-fd46b48f8dca None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:52.151764 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcdc9c-10d2-4e04-af93-fd46b48f8dca None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:52.152156 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcdc9c-10d2-4e04-af93-fd46b48f8dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:52.172564 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-78bcdc9c-10d2-4e04-af93-fd46b48f8dca None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:52.173853 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-78bcdc9c-10d2-4e04-af93-fd46b48f8dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:52.183494 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1464/2927] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:52.187889 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6f5f5-05cd-490b-8d0b-f602e2e9b5b3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:52.197450 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6f5f5-05cd-490b-8d0b-f602e2e9b5b3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:52.202567 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227025e0-6859-436e-a9bc-74c9946d6ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:52.203188 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227025e0-6859-436e-a9bc-74c9946d6ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:52.203460 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227025e0-6859-436e-a9bc-74c9946d6ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:52.203723 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227025e0-6859-436e-a9bc-74c9946d6ff8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:52.204360 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227025e0-6859-436e-a9bc-74c9946d6ff8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 16:33:52.204654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227025e0-6859-436e-a9bc-74c9946d6ff8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:52.206257 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227025e0-6859-436e-a9bc-74c9946d6ff8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:52.217966 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1464/2928] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 6 16:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 16:33:52.222281 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcdc9c-10d2-4e04-af93-fd46b48f8dca None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:52.232140 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcdc9c-10d2-4e04-af93-fd46b48f8dca None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:52.236865 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b44986-bb3f-4e2a-8520-e2feaf3b2da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:52.237462 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b44986-bb3f-4e2a-8520-e2feaf3b2da0 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:52.237737 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b44986-bb3f-4e2a-8520-e2feaf3b2da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:52.238005 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b44986-bb3f-4e2a-8520-e2feaf3b2da0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:52.238632 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b44986-bb3f-4e2a-8520-e2feaf3b2da0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:52.240643 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b44986-bb3f-4e2a-8520-e2feaf3b2da0 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:52.271569 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1465/2929] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 6 16:33:52 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:33:52.275988 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227025e0-6859-436e-a9bc-74c9946d6ff8 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:52.286048 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227025e0-6859-436e-a9bc-74c9946d6ff8 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:52.346738 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4eabe-7de0-4e38-a137-56fa57b4c07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:52.347347 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4eabe-7de0-4e38-a137-56fa57b4c07c admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:52.347626 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4eabe-7de0-4e38-a137-56fa57b4c07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:52.347930 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4eabe-7de0-4e38-a137-56fa57b4c07c admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:52.348631 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4eabe-7de0-4e38-a137-56fa57b4c07c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:52.350871 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4eabe-7de0-4e38-a137-56fa57b4c07c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:52.373537 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1465/2930] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 6 16:33:52 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 16:33:52.377770 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b44986-bb3f-4e2a-8520-e2feaf3b2da0 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:52.387914 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b44986-bb3f-4e2a-8520-e2feaf3b2da0 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:52.392433 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df737bc-3afd-4112-9c47-cb4e497f6ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:52.393015 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df737bc-3afd-4112-9c47-cb4e497f6ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:52.393286 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df737bc-3afd-4112-9c47-cb4e497f6ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:52.393552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df737bc-3afd-4112-9c47-cb4e497f6ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:52.394177 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df737bc-3afd-4112-9c47-cb4e497f6ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:52.395787 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df737bc-3afd-4112-9c47-cb4e497f6ac3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:52.405766 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1466/2931] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 16:33:52.409654 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4eabe-7de0-4e38-a137-56fa57b4c07c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:52.420300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4eabe-7de0-4e38-a137-56fa57b4c07c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:52.424998 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa754b4-b414-4e9d-89b4-bb7e1b0a05fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:52.425590 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa754b4-b414-4e9d-89b4-bb7e1b0a05fb admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:52.425870 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa754b4-b414-4e9d-89b4-bb7e1b0a05fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:52.426150 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa754b4-b414-4e9d-89b4-bb7e1b0a05fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:52.426977 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa754b4-b414-4e9d-89b4-bb7e1b0a05fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:52.428736 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa754b4-b414-4e9d-89b4-bb7e1b0a05fb admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:52.440797 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1466/2932] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 6 16:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 16:33:52.444912 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df737bc-3afd-4112-9c47-cb4e497f6ac3 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:52.454831 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df737bc-3afd-4112-9c47-cb4e497f6ac3 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:52.459372 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be88267-b72d-48a6-82c2-fec1304c6f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:52.460002 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be88267-b72d-48a6-82c2-fec1304c6f81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:52.460274 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be88267-b72d-48a6-82c2-fec1304c6f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:52.460794 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be88267-b72d-48a6-82c2-fec1304c6f81 admin admin] PATH_INFO: `/v3/projects/c372c59018de49789d57310f99e20cb3/users/eeedda0df86a48df8f83a20bd8668c15/roles/e89162686187404da84651e0a3632b7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:52.475228 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be88267-b72d-48a6-82c2-fec1304c6f81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c372c59018de49789d57310f99e20cb3, user_id=eeedda0df86a48df8f83a20bd8668c15, role_id=e89162686187404da84651e0a3632b7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:52.477837 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be88267-b72d-48a6-82c2-fec1304c6f81 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:52.491405 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1467/2933] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 6 16:33:52 2022] PUT /identity/v3/projects/c372c59018de49789d57310f99e20cb3/users/eeedda0df86a48df8f83a20bd8668c15/roles/e89162686187404da84651e0a3632b7b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:33:52.503722 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b01fc-45ad-44cd-bb72-1f89ab93d91d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:52.504030 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b01fc-45ad-44cd-bb72-1f89ab93d91d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:52.504300 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b01fc-45ad-44cd-bb72-1f89ab93d91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:52.532807 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-cb1b01fc-45ad-44cd-bb72-1f89ab93d91d None None] MFA Rules not processed for user `eeedda0df86a48df8f83a20bd8668c15`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:33:52.534078 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cb1b01fc-45ad-44cd-bb72-1f89ab93d91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:52.600517 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1467/2934] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:33:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 16:33:52.611230 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be88267-b72d-48a6-82c2-fec1304c6f81 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:33:52.612894 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8be88267-b72d-48a6-82c2-fec1304c6f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:33:52.635413 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:33:52.672223 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be88267-b72d-48a6-82c2-fec1304c6f81 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:33:52.675780 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fbdb7a-8683-4d66-b5c4-a815b78608b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7849ef049ffd4495909f997735af8782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:33:52.676376 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbdb7a-8683-4d66-b5c4-a815b78608b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:33:52.676649 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbdb7a-8683-4d66-b5c4-a815b78608b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:33:52.676915 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbdb7a-8683-4d66-b5c4-a815b78608b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:33:52.685898 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fbdb7a-8683-4d66-b5c4-a815b78608b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:33:52.687331 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fbdb7a-8683-4d66-b5c4-a815b78608b7 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:33:52.695915 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1468/2935] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:33:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:34:02.597403 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:34:02.691951 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=17 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:34:16.255793 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b01fc-45ad-44cd-bb72-1f89ab93d91d None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:34:16.319375 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b01fc-45ad-44cd-bb72-1f89ab93d91d None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:34:16.323950 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b90d73-65c0-4673-ad96-7b538e73398c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:34:16.324577 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b90d73-65c0-4673-ad96-7b538e73398c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:34:16.324854 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b90d73-65c0-4673-ad96-7b538e73398c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:34:16.325123 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b90d73-65c0-4673-ad96-7b538e73398c admin admin] PATH_INFO: `/v3/users/eeedda0df86a48df8f83a20bd8668c15` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:34:16.326260 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b90d73-65c0-4673-ad96-7b538e73398c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeedda0df86a48df8f83a20bd8668c15)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:34:16.328201 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b90d73-65c0-4673-ad96-7b538e73398c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:34:16.375638 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-68b90d73-65c0-4673-ad96-7b538e73398c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeedda0df86a48df8f83a20bd8668c15'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:34:16.382676 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-68b90d73-65c0-4673-ad96-7b538e73398c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeedda0df86a48df8f83a20bd8668c15'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:34:16.395095 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-68b90d73-65c0-4673-ad96-7b538e73398c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeedda0df86a48df8f83a20bd8668c15'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:34:16.395800 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-68b90d73-65c0-4673-ad96-7b538e73398c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeedda0df86a48df8f83a20bd8668c15'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:34:16.403163 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-68b90d73-65c0-4673-ad96-7b538e73398c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeedda0df86a48df8f83a20bd8668c15'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:34:16.409655 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1468/2936] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:34:16 2022] DELETE /identity/v3/users/eeedda0df86a48df8f83a20bd8668c15 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:34:16.414185 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9be8b-a3c1-409f-b1ea-cacb312bae89 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:34:16.414731 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9be8b-a3c1-409f-b1ea-cacb312bae89 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:34:16.414850 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9be8b-a3c1-409f-b1ea-cacb312bae89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:34:16.442630 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0ec9be8b-a3c1-409f-b1ea-cacb312bae89 None None] MFA Rules not processed for user `feda116f177c415fa220633084d10d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 16:34:16.443899 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0ec9be8b-a3c1-409f-b1ea-cacb312bae89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:34:16.503059 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1469/2937] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 6 16:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 16:34:16.512598 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b90d73-65c0-4673-ad96-7b538e73398c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:34:16.514007 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-68b90d73-65c0-4673-ad96-7b538e73398c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:34:16.575543 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b90d73-65c0-4673-ad96-7b538e73398c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:34:16.579341 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8e055-3c3d-449a-9548-f631934e7c56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:34:16.580515 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8e055-3c3d-449a-9548-f631934e7c56 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:34:16.580806 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8e055-3c3d-449a-9548-f631934e7c56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:34:16.581089 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8e055-3c3d-449a-9548-f631934e7c56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:34:16.591724 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8e055-3c3d-449a-9548-f631934e7c56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:34:16.593226 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8e055-3c3d-449a-9548-f631934e7c56 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:34:16.603878 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1469/2938] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:34:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:34:16.880525 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9be8b-a3c1-409f-b1ea-cacb312bae89 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:34:16.882036 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0ec9be8b-a3c1-409f-b1ea-cacb312bae89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:34:16.896308 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9be8b-a3c1-409f-b1ea-cacb312bae89 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:34:16.901524 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea232df4-9b83-4ec4-ac93-7bff046ff68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:34:16.902262 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea232df4-9b83-4ec4-ac93-7bff046ff68c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:34:16.903448 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea232df4-9b83-4ec4-ac93-7bff046ff68c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:34:16.903772 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea232df4-9b83-4ec4-ac93-7bff046ff68c admin admin] PATH_INFO: `/v3/projects/c372c59018de49789d57310f99e20cb3` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:34:16.905030 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea232df4-9b83-4ec4-ac93-7bff046ff68c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c372c59018de49789d57310f99e20cb3)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:34:16.907079 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea232df4-9b83-4ec4-ac93-7bff046ff68c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:34:16.957021 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ea232df4-9b83-4ec4-ac93-7bff046ff68c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c372c59018de49789d57310f99e20cb3', 'request_id': 'req-ea232df4-9b83-4ec4-ac93-7bff046ff68c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:34:16.965802 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ea232df4-9b83-4ec4-ac93-7bff046ff68c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c372c59018de49789d57310f99e20cb3', 'request_id': 'req-ea232df4-9b83-4ec4-ac93-7bff046ff68c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:34:16.972316 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ea232df4-9b83-4ec4-ac93-7bff046ff68c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c372c59018de49789d57310f99e20cb3', 'request_id': 'req-ea232df4-9b83-4ec4-ac93-7bff046ff68c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:34:16.983355 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ea232df4-9b83-4ec4-ac93-7bff046ff68c admin admin] The token cache is being invalidate because project c372c59018de49789d57310f99e20cb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:34:16.983643 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ea232df4-9b83-4ec4-ac93-7bff046ff68c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:34:16.984287 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ea232df4-9b83-4ec4-ac93-7bff046ff68c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:34:16.985779 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1470/2939] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:34:16 2022] DELETE /identity/v3/projects/c372c59018de49789d57310f99e20cb3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:34:26.598911 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:34:26.980320 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:36.891877 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8e055-3c3d-449a-9548-f631934e7c56 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:36.893444 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-48f8e055-3c3d-449a-9548-f631934e7c56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:37.016430 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8e055-3c3d-449a-9548-f631934e7c56 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:37.062700 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea232df4-9b83-4ec4-ac93-7bff046ff68c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:37.064624 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ea232df4-9b83-4ec4-ac93-7bff046ff68c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:37.102604 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d080bec-8f48-4b70-96fe-2ed3e6dac346 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6afd4f6ffdae4b16aa5ba757e68c18c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:37.103489 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d080bec-8f48-4b70-96fe-2ed3e6dac346 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:37.103758 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d080bec-8f48-4b70-96fe-2ed3e6dac346 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:37.103953 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d080bec-8f48-4b70-96fe-2ed3e6dac346 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:37.105830 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1d080bec-8f48-4b70-96fe-2ed3e6dac346 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:37.119896 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d080bec-8f48-4b70-96fe-2ed3e6dac346 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:37.121512 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d080bec-8f48-4b70-96fe-2ed3e6dac346 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:37.130812 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea232df4-9b83-4ec4-ac93-7bff046ff68c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:37.133925 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1470/2940] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:37.135403 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c17134-acfe-4163-88a2-1bbf3b514845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:37.136156 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c17134-acfe-4163-88a2-1bbf3b514845 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:37.136427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c17134-acfe-4163-88a2-1bbf3b514845 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:37.136759 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c17134-acfe-4163-88a2-1bbf3b514845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:37.138843 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-55c17134-acfe-4163-88a2-1bbf3b514845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:37.153999 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c17134-acfe-4163-88a2-1bbf3b514845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:37.155574 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c17134-acfe-4163-88a2-1bbf3b514845 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:37.168429 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1471/2941] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:35:38.250874 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d080bec-8f48-4b70-96fe-2ed3e6dac346 service nova] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:38.261184 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d080bec-8f48-4b70-96fe-2ed3e6dac346 service nova] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:38.264812 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c018800-74e2-4a35-a0b5-13937a42fcbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:38.265413 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c018800-74e2-4a35-a0b5-13937a42fcbf service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:38.265684 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c018800-74e2-4a35-a0b5-13937a42fcbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:38.265955 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c018800-74e2-4a35-a0b5-13937a42fcbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:38.267765 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5c018800-74e2-4a35-a0b5-13937a42fcbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:38.267968 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c17134-acfe-4163-88a2-1bbf3b514845 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:38.277902 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c018800-74e2-4a35-a0b5-13937a42fcbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:38.278129 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c17134-acfe-4163-88a2-1bbf3b514845 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:38.279384 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c018800-74e2-4a35-a0b5-13937a42fcbf service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:38.281655 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0484e3d-6e5e-4f07-bd65-1944a13b787b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:38.282253 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0484e3d-6e5e-4f07-bd65-1944a13b787b service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:38.282548 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0484e3d-6e5e-4f07-bd65-1944a13b787b service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:38.282980 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0484e3d-6e5e-4f07-bd65-1944a13b787b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:38.287977 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1471/2942] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:38.291600 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0484e3d-6e5e-4f07-bd65-1944a13b787b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:38.293066 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0484e3d-6e5e-4f07-bd65-1944a13b787b service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:38.301663 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1472/2943] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 16:35:38.374230 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c018800-74e2-4a35-a0b5-13937a42fcbf service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:38.375767 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5c018800-74e2-4a35-a0b5-13937a42fcbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:38.386962 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c018800-74e2-4a35-a0b5-13937a42fcbf service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:38.390563 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2d17c-66c5-43f7-a5c2-ab4e8515e3fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:38.391174 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2d17c-66c5-43f7-a5c2-ab4e8515e3fe service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:38.391447 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2d17c-66c5-43f7-a5c2-ab4e8515e3fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:38.391714 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2d17c-66c5-43f7-a5c2-ab4e8515e3fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:38.393402 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-78b2d17c-66c5-43f7-a5c2-ab4e8515e3fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:38.406773 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2d17c-66c5-43f7-a5c2-ab4e8515e3fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:38.408189 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2d17c-66c5-43f7-a5c2-ab4e8515e3fe service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:38.419765 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1472/2944] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:48.297644 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=32 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:48.415027 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=86 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:51.244827 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0484e3d-6e5e-4f07-bd65-1944a13b787b service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:51.253668 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0484e3d-6e5e-4f07-bd65-1944a13b787b service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:51.257427 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b41c6-8a3b-4f9c-9654-11c74a95b8fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '989beaca146d4f9b950f333bd3567e6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:51.258055 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b41c6-8a3b-4f9c-9654-11c74a95b8fb service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:51.259051 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b41c6-8a3b-4f9c-9654-11c74a95b8fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:51.259326 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b41c6-8a3b-4f9c-9654-11c74a95b8fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:51.261124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e10b41c6-8a3b-4f9c-9654-11c74a95b8fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:51.325017 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b41c6-8a3b-4f9c-9654-11c74a95b8fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:51.326445 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b41c6-8a3b-4f9c-9654-11c74a95b8fb service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:51.336155 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1473/2945] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:54.456053 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2d17c-66c5-43f7-a5c2-ab4e8515e3fe service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:54.457447 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-78b2d17c-66c5-43f7-a5c2-ab4e8515e3fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:54.553671 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2d17c-66c5-43f7-a5c2-ab4e8515e3fe service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:54.561396 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941cbe0-beff-461f-80b3-d967078a9a71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034d75ab747e4c0cab040e47c6a6236e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:54.562269 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941cbe0-beff-461f-80b3-d967078a9a71 service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:54.562791 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941cbe0-beff-461f-80b3-d967078a9a71 service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:54.566618 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941cbe0-beff-461f-80b3-d967078a9a71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:54.573581 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1941cbe0-beff-461f-80b3-d967078a9a71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:54.589025 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941cbe0-beff-461f-80b3-d967078a9a71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:54.590823 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941cbe0-beff-461f-80b3-d967078a9a71 service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:54.607603 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1473/2946] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:35:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:35:57.497103 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b41c6-8a3b-4f9c-9654-11c74a95b8fb service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:57.498903 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e10b41c6-8a3b-4f9c-9654-11c74a95b8fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:57.518853 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b41c6-8a3b-4f9c-9654-11c74a95b8fb service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:57.523729 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb3417-b39f-4387-b773-224d2914b1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:57.524818 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb3417-b39f-4387-b773-224d2914b1f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:57.525179 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb3417-b39f-4387-b773-224d2914b1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:57.526044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb3417-b39f-4387-b773-224d2914b1f9 admin admin] PATH_INFO: `/v3/users/210d0f461d2c445a9688bc233d048079` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:57.527249 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb3417-b39f-4387-b773-224d2914b1f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=210d0f461d2c445a9688bc233d048079)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:57.529048 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb3417-b39f-4387-b773-224d2914b1f9 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:57.581693 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-04cb3417-b39f-4387-b773-224d2914b1f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '210d0f461d2c445a9688bc233d048079'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:57.588946 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-04cb3417-b39f-4387-b773-224d2914b1f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '210d0f461d2c445a9688bc233d048079'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:57.600734 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-04cb3417-b39f-4387-b773-224d2914b1f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '210d0f461d2c445a9688bc233d048079'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:57.601447 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-04cb3417-b39f-4387-b773-224d2914b1f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '210d0f461d2c445a9688bc233d048079'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:57.602567 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-04cb3417-b39f-4387-b773-224d2914b1f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '210d0f461d2c445a9688bc233d048079'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:57.615552 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1474/2947] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:35:57 2022] DELETE /identity/v3/users/210d0f461d2c445a9688bc233d048079 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 16:35:57.795161 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941cbe0-beff-461f-80b3-d967078a9a71 service placement] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:57.807271 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941cbe0-beff-461f-80b3-d967078a9a71 service placement] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:57.865967 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef63c56-b64b-4353-b310-c1df78af0441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:57.866590 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef63c56-b64b-4353-b310-c1df78af0441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:57.866897 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef63c56-b64b-4353-b310-c1df78af0441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:57.867163 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef63c56-b64b-4353-b310-c1df78af0441 admin admin] PATH_INFO: `/v3/projects/870b3a5a8fc74957bf004f0a6f6020a3` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:57.868814 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef63c56-b64b-4353-b310-c1df78af0441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=870b3a5a8fc74957bf004f0a6f6020a3)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:57.870727 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef63c56-b64b-4353-b310-c1df78af0441 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:57.917336 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-1ef63c56-b64b-4353-b310-c1df78af0441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '870b3a5a8fc74957bf004f0a6f6020a3', 'request_id': 'req-1ef63c56-b64b-4353-b310-c1df78af0441'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:57.923422 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-1ef63c56-b64b-4353-b310-c1df78af0441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '870b3a5a8fc74957bf004f0a6f6020a3', 'request_id': 'req-1ef63c56-b64b-4353-b310-c1df78af0441'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:57.929158 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-1ef63c56-b64b-4353-b310-c1df78af0441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '870b3a5a8fc74957bf004f0a6f6020a3', 'request_id': 'req-1ef63c56-b64b-4353-b310-c1df78af0441'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:57.942926 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-1ef63c56-b64b-4353-b310-c1df78af0441 admin admin] The token cache is being invalidate because project 870b3a5a8fc74957bf004f0a6f6020a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:35:57.943220 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-1ef63c56-b64b-4353-b310-c1df78af0441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:57.943876 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-1ef63c56-b64b-4353-b310-c1df78af0441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:57.945499 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1474/2948] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:35:57 2022] DELETE /identity/v3/projects/870b3a5a8fc74957bf004f0a6f6020a3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:57.950273 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb3417-b39f-4387-b773-224d2914b1f9 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:57.951709 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-04cb3417-b39f-4387-b773-224d2914b1f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:57.965124 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb3417-b39f-4387-b773-224d2914b1f9 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:58.022687 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15904d79-0a8f-4ff9-9a7c-b1af0ffac921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:58.023319 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15904d79-0a8f-4ff9-9a7c-b1af0ffac921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:58.023598 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15904d79-0a8f-4ff9-9a7c-b1af0ffac921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:58.023951 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15904d79-0a8f-4ff9-9a7c-b1af0ffac921 admin admin] PATH_INFO: `/v3/users/fd9ff285c2c8473faa96355a927bc4e7` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:58.025044 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15904d79-0a8f-4ff9-9a7c-b1af0ffac921 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd9ff285c2c8473faa96355a927bc4e7)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:58.026995 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15904d79-0a8f-4ff9-9a7c-b1af0ffac921 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:58.032388 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:58.074440 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-15904d79-0a8f-4ff9-9a7c-b1af0ffac921 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd9ff285c2c8473faa96355a927bc4e7'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:58.081265 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-15904d79-0a8f-4ff9-9a7c-b1af0ffac921 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd9ff285c2c8473faa96355a927bc4e7'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:58.093304 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-15904d79-0a8f-4ff9-9a7c-b1af0ffac921 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd9ff285c2c8473faa96355a927bc4e7'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:58.093972 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-15904d79-0a8f-4ff9-9a7c-b1af0ffac921 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd9ff285c2c8473faa96355a927bc4e7'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:58.094570 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-15904d79-0a8f-4ff9-9a7c-b1af0ffac921 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd9ff285c2c8473faa96355a927bc4e7'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:58.107852 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1475/2949] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 6 16:35:57 2022] DELETE /identity/v3/users/fd9ff285c2c8473faa96355a927bc4e7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:35:58.284700 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef63c56-b64b-4353-b310-c1df78af0441 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:35:58.286206 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1ef63c56-b64b-4353-b310-c1df78af0441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:35:58.301021 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef63c56-b64b-4353-b310-c1df78af0441 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:35:58.319826 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:35:58.362799 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ecd283-1993-4bb8-8e02-14ca110fca49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feda116f177c415fa220633084d10d2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcbe5a1252a54f0a914b3779a09b044d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:35:58.363426 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecd283-1993-4bb8-8e02-14ca110fca49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:35:58.363703 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecd283-1993-4bb8-8e02-14ca110fca49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:35:58.363976 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecd283-1993-4bb8-8e02-14ca110fca49 admin admin] PATH_INFO: `/v3/projects/082bb335eea440d0a1145339ee9610aa` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:35:58.365100 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ecd283-1993-4bb8-8e02-14ca110fca49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=082bb335eea440d0a1145339ee9610aa)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:35:58.367033 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ecd283-1993-4bb8-8e02-14ca110fca49 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:35:58.415718 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-33ecd283-1993-4bb8-8e02-14ca110fca49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '082bb335eea440d0a1145339ee9610aa', 'request_id': 'req-33ecd283-1993-4bb8-8e02-14ca110fca49'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:58.421842 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-33ecd283-1993-4bb8-8e02-14ca110fca49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '082bb335eea440d0a1145339ee9610aa', 'request_id': 'req-33ecd283-1993-4bb8-8e02-14ca110fca49'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:58.427389 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-33ecd283-1993-4bb8-8e02-14ca110fca49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '082bb335eea440d0a1145339ee9610aa', 'request_id': 'req-33ecd283-1993-4bb8-8e02-14ca110fca49'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:58.440327 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-33ecd283-1993-4bb8-8e02-14ca110fca49 admin admin] The token cache is being invalidate because project 082bb335eea440d0a1145339ee9610aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 16:35:58.440613 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-33ecd283-1993-4bb8-8e02-14ca110fca49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:58.441220 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-33ecd283-1993-4bb8-8e02-14ca110fca49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 16:35:58.442720 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1475/2950] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 6 16:35:58 2022] DELETE /identity/v3/projects/082bb335eea440d0a1145339ee9610aa => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 16:36:08.104346 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:36:08.437424 dv-cinder-852208-18-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 16:38:37.327877 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15904d79-0a8f-4ff9-9a7c-b1af0ffac921 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 16:38:37.329322 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-15904d79-0a8f-4ff9-9a7c-b1af0ffac921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:38:37.399607 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15904d79-0a8f-4ff9-9a7c-b1af0ffac921 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 16:38:37.461526 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd217d3-d6d0-4307-8357-2babdd671b0a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '034d75ab747e4c0cab040e47c6a6236e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0138d62d5879495499a17e20310d4f61', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 16:38:37.462137 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd217d3-d6d0-4307-8357-2babdd671b0a service placement] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 16:38:37.462419 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd217d3-d6d0-4307-8357-2babdd671b0a service placement] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 16:38:37.462728 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd217d3-d6d0-4307-8357-2babdd671b0a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 16:38:37.464442 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ecd217d3-d6d0-4307-8357-2babdd671b0a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 16:38:37.477172 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd217d3-d6d0-4307-8357-2babdd671b0a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 16:38:37.478895 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd217d3-d6d0-4307-8357-2babdd671b0a service placement] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 16:38:37.489351 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1476/2951] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 6 16:38:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 16:38:47.485080 dv-cinder-852208-18-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=58 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}